I posted an update¹ regarding my home server on ESXi a few months ago, and that I was re-evaluating how my data storage implementation was going to change. I’m happy to report that I’ve finally found an appropriate solution, utilizing the NexentaStor OpenSolaris-based OS². While not without its problems (notably a buggy web-management interface, I suspect due to Python issues), it has proved itself to be quite stable as far as data access and integrity.

In my last post about my home server I mentioned a few issues I would have migrating data, including the scaling down an LVM-based Linux VM to enable RDM (Raw Disk Mapping) to the NexentaStor VM. I mitigated this issue by adding 3 new drives, one ATA based 500GB drive to host the actual virtual machine installations, and two new 1.5TB SATA drives for storage. I mapped these two new drives to the NexentaStor VM using RDM and set them up as a ZFS array. I then migrated data over the network from the old Ubuntu VM to the new storage VM. After a large portion of data was migrated, I diminished the size of the LVM to free up the 1.5TB drive, and disconnected from the old storage VM. I then mapped it to the new storage VM and added it to the ZFS array. I then migrated the rest of the data and deleted the old storage VM.

So, my data storage needs are met for now. Unfortunately, the original 1TB drive has the ESXi installation on it, meaning I can’t format and add to the new ZFS array. I am considering the best approach to this, but it seems I’ll likely move the VM’s from the 500GB drive to the 1TB, disconnect all drives except the 500, and install ESXi to the drive. After this I would import the machines from the 1TB back to the 500GB drive. My concern with this approach is how to ensure that the physical ZFS disks import appropriately to the storage VM.

As always, post any questions in the comment section, or send me an email, and I’ll do my best to respond.

Related posts:

1. ESXi, ZFS, and Home Server Data Storage – Part 1
2. VMWare ESXi as a Media Server
3. Update Re: Print Issues

The point I’d like to make here is that Google may be phasing out Windows 7, not because of any inherent security flaws, but that it’s possible that licensing and maintenance costs are the driving factor to phasing out Windows. Perhaps Google feels that it could spend less effort maintaining internal security by using a Linux or Unix-based OS.

Related posts:

1. More on Client Platform Security
2. Google Wave
3. On the Security of Client Platforms

Regarding Foxconn Suicides:
Steve: It’s a factory — but my gosh, they have restaurants and movie theaters… but it’s a factory. But they’ve had some suicides and attempted suicides — and they have 400,000 people there. The rate is under what the US rate is, but it’s still troubling.
Steve: We had this in my hometown of Palo Alto, copy cat suicides.

Regarding Search:
Walt: So last year we had a company called Siri, a search company…
Steve: They’re not a search company. They’re an AI company. We have no plans to go into the search business. We don’t care about it — other people do it well.

Regarding Press:
Steve: Well I think the foundation of a free society is a free press. And we’ve seen what’s happening to papers in the US right now. I think they’re really important. I don’t want to see us descend into a nation of bloggers. I think we need editors now more than ever.
Steve: I can tell you as one of the largest sellers of content on the internet to date — price it aggressively and go for volume. That has worked for us. I’m trying to get the press to do the same thing. They need to do it differently than they do it for print.
Steve: I think people are willing to pay for content. I believe it for music and video, and I believe it for the media.

Regarding Interaction with Computing Devices:
Steve: People laugh at me when I say it’s magical… but something has been stripped away and removed between you and the computer…

Regarding Pulitzer-winning Cartoon Censorship:
Steve: Yes… and political cartoons got caught in that. We didn’t think of that. So this guy submits his app and he gets rejected. We didn’t see that coming. So we changed the rule, but this guy never resubmitted… then he wins a Pulitzer Prize, and he says we rejected him. So, we are guilty of making mistakes. We’re doing the best we can, we’re learning as fast as we can — but we thought this rule made sense.
Steve: We’re doing the best we can, we’re fixing mistakes. But what happens is — people lie. And then they run to the press and tell people about this oppression, and they get their 15 minutes of fame. We don’t run to the press and say “this guy is a son of a bitch liar!” — we don’t do that.
Steve: 95% (of app submissions) are approved within 7 days…

Regarding Gizmodo:
Steve: Well a guy… who can say if he’s a journalist.
Steve: The person who took the phone plugged it into his roommates computer. And this guy was trying to destroy evidence… and his roommate called the police. So this is a story that’s amazing — it’s got theft, it’s got buying stolen property, it’s got extortion, I’m sure there’s some sex in there (huge laughs)… the whole thing is very colorful. The DA is looking into it, and to my knowledge they have someone making sure they only see stuff that relates to this case. I don’t know how it will end up.
Steve: You know, when this whole thing with Gizmodo happened, I got advice from people who said ‘you gotta just let it slide, you shouldn’t go after a journalist just because they bought stolen property and tried to extort you.’ And I thought deeply about this, and I concluded the worst thing that could happen is if we change our core values and let it slide. I can’t do that. I’d rather quit.

Regarding TOS Change for Analytics:
Q: You changed your TOS on analytics — do you want to own that for developer’s data?
A: Well we learned this really interesting thing. Some company called Flurry had data on devices that we were using on our campus — new devices. They were getting this info by getting developers to put software in their apps that sent info back to this company! So we went through the roof. It’s violating our privacy policies, and it’s pissing us off! So we said we’re only going to allow analytics that don’t give our device info — only for the purpose of advertising. But you can’t get info off of our devices and turn around and sell it. That you can’t do… is that clear?
Q: It is, but there is vital info there that could make apps better…
A: That’s true… but there’s no excuse for them not asking customers if it’s okay to send that data. We’re willing to talk to some of these people when we calm down… but it’s not today.

Regarding DRM-ed Content:
Q: I bought the movie Up on DVD, it had a digital download. I put it on my iPad. I hooked up my VGA adapter and tried to play it… but I couldn’t because of HDCP. Can you tell me how you’re helping with this?
A: We didn’t invent this stuff…
Q: But you did deploy it…
A: Well the content creators are trying to protect this stuff, and they’re grabbing at straws. Sometimes they grab the right ones, and sometimes they don’t. If we want access to this stuff, we have to play by some of their rules. I feel your pain.

Regarding TV:
Q: Hi, I’m from Hillcrest Labs… do you think it’s time to throw out the interface for TV? When will Apple do something there?
A: The problem with innovation in the TV industry is the go to market strategy. The TV industry has a subsidized model that gives everyone a set top box for free. So no one wants to buy a box. Ask TiVo, ask Roku, ask us… ask Google in a few months. So all you can do is ADD a box to the TV. You just end up with a table full of remotes, a cluster of boxes… and that’s what we have today. The only way that’s going to change is if you tear up the set top box, give it a new UI, and get it in front of consumers in a way they’re going to want it. The TV is going to lose in our eyes until there is a better go to market strategy… otherwise you’re just making another TiVo.
Q: In the phone area you were able to partner with a carrier… would you do that with TV?
A: Well then there’s a problem, providers are local… it’s a Tower of Babel problem…

Related posts:

1. Apple iPad
2. Extracting Exchange Contacts from iPhone

I don’t trust Facebook. Never have, and never will. Unfortunately, with their actions they’ve lowered my trust from minimal levels to less than zero.

Unfortunately, they’ve got me by the cajones. Their client base includes my connections. I have to decide whether my distrust of Facebook is great enough to sever my easy access of up to date information from those close to me. How important is that data to me?

Related posts:

1. On the use of Facebook

UPDATE: It seems I may be re-evaluating my choice of OpenSolaris vs. Nexenta. I’ve experienced a lot of issues with SMB/CIFS authentication on OpenSolaris, and have not been able to get it to work properly. I’ve also had a reply from a commenter assuring me as to the stability of Nexenta 3. I’ll post again once I’ve re-evaluated my choice of SAN OS.

A few months ago I wrote¹ about a new home server I was setting up. I designed the server from ground up to handle VMware ESXi 4.0. When I built it I did not build in data redundancy, as I had two mismatched drives (a 1.5TB and a 1TB). Also, because I was relatively new to ESXi, I created the datastore with default block sizes, limiting me to 256GB virtual disk file sizes. I used Ubuntu Linux to link the virtual disks together with Logical Volume Manager (LVM), and create one big mount for my data storage. Unfortunately, the 2.5TB volume is now full.

With a full server volume, nothing would seem more obvious than to go out and buy more storage. So, I went out and bought two 1.5TB disks (I don’t believe that 2TB drive sizes have hit the optimal price point yet). On my way home however, I realized that I now have the capacity for almost 5.5TB of storage. If one physical drive were to crash, I would not only lose the data that was on that drive, I would also lose my entire dataset. LVM does not handle missing drives, so the entire logical volume, with all my data on it, would be gone. This is far too much risk, and I decided to build in data redundancy.

With that in mind, I began to consider various options. The motherboard’s BIOS supports RAID 1,0, 10, and 5. Of those options, I would prefer a RAID5 configuration, as it offers the best capacity/redundancy ratio. Unfortunately however, I’ve already got ESXi installed on the existing 1.5TB drive, and the data between it and the second drive must remain intact. I don’t know how well ESXi would handle a sudden BIOS change to a RAID configuration. Also, after some reading, I found that it was likely that drivers would be required in the OS install to support the RAID configuration. There are too many unknown variables to risk my data with a BIOS RAID configuration change.

The next option I considered was a software level RAID5 implementation, one where I’d have a virtual machine handle the RAID5 control. Unfortunately however, this approach also has its drawbacks. RAID5 requires 3 drives of the same size to setup. I have 3 1.5TB drives right now, but one of them is full of data, including my ESXi host install. I would have to create a deteriorated RAID5 array with two drives, install another physical drive for the ESXi host install, import my original ESXi host configuration to the new host install, move my data to the new array, then move the actual client OS virtual disk to the new physical drive. After that point I could wipe the original 1.5TB and add it to the RAID5 array. I would be left with the 1TB to use for other purposes. During this whole process praying that something does not mess up the LVM i the Linux install. All in all, a very messy endeavor. Too much risk, both with the data itself as well as with the host/client OS installations.

Since a RAID configuration seemed to be out, I looked for other ‘outside-the-box’ solutions. Obviously it would have to be a disk/file level solution, as LVM with virtual disks wasn’t going to cut it. Then I remembered looking at ZFS² (a file system format) a couple years ago. ZFS offers great data redundancy for little disk cost, flexibility, compression, good performance, and a host of other things (things most non-technical people wouldn’t care about). The stability of the filesystem has come a long way since I first looked at it (it was more proof of concept at the time), to the point where I would trust my data with it. ZFS seemed to fit my current needs and network conditions perfectly.

Now that I’d decided on ZFS as my new network storage solution, I had to decide how I was going to implement it. Because it was developed by SUN Microsystems, there are licensing quirks that have kept it from being incorporated into the Linux kernel. There is however, an implementation via the FUSE project. I could potentially install it into my Ubuntu media server virtual machine, and have a relatively easy transition. After some investigation however, I felt that ZFS-fuse was still too much of a hack for me to trust my data with.

The only other real ZFS options were FreeBSD, OpenSolaris, and a project called Nexenta³. Nexenta is a Gnome (Ubuntu-like) user land built around the OpenSolaris kernel. This initially attracted me quite a bit, as it seemed to perhaps be the easiest way forward. Two things kept me back however. One, the version of Nexenta that offers deduplication support for ZFS is currently labeled beta. Two, because it was built around the OpenSolaris kernel, there would be a lot more hacking required if I was going to try to replicate my Ubuntu media serving services. At this point I realized that it would be easiest to keep my Ubuntu media server, and just point its data volumes to another VM’s network share, as though the other VM was a SAN. So, I decided that Nexenta was more than I needed, and that it was targeting a different person than I. Add to that the lack of deduplication, and Nexenta was out.

The other two ZFS options were OpenSolaris and FreeBSD. Since I’m a Max OS X (built around BSD Unix) power user, it seemed the most attractive option. On doing some analysis however, it seemed that OpenSolaris had better support and a better-performing ZFS implementation. Consequently, I’ve decided to go the OpenSolaris route for my virtual SAN.

I’ll post Part 2 over the next week or so. Part 2 will cover the actual implementation (still in progress), and some of the challenges encountered.

Related posts:

1. Update re: ESXi & ZFS Storage
2. VMWare ESXi as a Media Server

Fortunately the Spirit jailbreak² tool was released just after the launch of the 3G iPad this past weekend. The Spirit jailbreak uses a common exploit to the iPad and iPhone to jailbreak, and consequently, I was able to jailbreak my iPhone 3GS running the latest firmware/baseband. I installed MyWi³ a tethering application that goes beyond Apple’s Bluetooth/USB tethering to provide tethering via WiFi, similar to the Verizon MiFi. I set up a WiFi network and password protected it. Thanks to the iPad’s network memory feature, every time I launch MyWi, it starts the protected network, and the iPad automatically connects. Smooth and seamless.

The only problem I’ve had with this setup is the bandwidth usage. In 40 minutes I used up 33Mb down and 6.5 up. This was just doing basic web browsing, email, and a speed test. I don’t know how tenable this usage is on my 6GB/mo plan.

Here’s a screenshot of my bandwidth up and down in Toronto on Rogers 3G network.

Related posts:

1. Thoughts on the iPad
2. Microsoft vs. iPad
3. Apple iPad

The reason I suggest that IT pros study design principles is so that they learn to appreciate the ‘user-level’ features for what they are, not dismissing them as merely useless pieces of eye-candy. When a tech can look at an object or interface, and not allow their ‘function over form’ mentality to get the better of them, they can better relate to how the user will experience the interface. When the tech appreciates on some level what the user appreciates, understanding between the two parties can grow.

Hand-in-hand with the previous point, IT pros should also study user interface experience. By beginning to understand how to implement or at least structure a good user experience, the eyes are opened to a new perspective on how the world functions. Studying and ingraining good UX principles in the core of the tech’s mentality will drastically shift how they approach their users, and how they approach the potential technologies to be implemented.

Now, you may ask why I’m suggesting the actual techs learn these things instead of just ensuring that the head folks know and understand these things. I’m a firm believer in the ancient Greek holistic approach to education. However, beyond that, I think that while the head of the body is responsible for the rest of a person’s functionality, when the whole body is trained and tuned to the same level, the performance level of the particular body jumps exponentially. Consequently, new and previously unattainable heights can be reached.

If you’re interested in some design oriented websites, check out these for examples (mostly centered around web design):

Smashing Magazine: http://www.smashingmagazine.com
Noupe: http://www.noupe.com
Net-Tuts: http://net.tutsplus.com

Related posts:

1. Apple iPad

I refused to write a review on the iPad in the first week. In my mind, the iPad is a paradigm shifting technology. Until I’d used the iPad for a few weeks, I did not feel I could provide the perspective I feel is important to my readers. Reviewing a new device is like a relationship. All fuzzy feelings for the first little while, then reality hits. I wanted to write a review that was not affected by initial emotions or excitement.

So, without further ado, let me tell you my experience with the iPad.

Battery life. This thing is phenomenal. I have never had another electronic device that did not make me nervous toward the end of a day of use. Even my iPhone 3GS does not provide me with the same sense of security that the device will not die by the end of a day of use. I can even charge the iPad Friday morning, take it up north (always careful with how I use it when away from a charger), and still have 40% battery left. I completely trust the iPad to be charged when I need it.

Convenience. Portability. The iPad has changed the frequency with which I take my computer with me. I rarely take my MacBook Pro from home anymore. I take the iPad everywhere. It’s like carrying a notebook with me, weight-wise. I pick it up off the desk and carry it in my hand. I don’t need a backpack to take my computer with me. This has changed my perspective on using my personal commuting device. It has become a permanent companion, rather than a tool to be utilized when needed. Similar to my iPhone.

I have begun to reevaluate my need for the MacBook Pro. All my data is stored on my Ubuntu mediaserver (hosted on VMware ESXi along with my Windows VM’s). I now utilize the iMac for managing my virtual servers and ESXi. I have Air Video Server installed for streaming movies from the media server to the iPad, and FileBrowser installed for browsing the network files from the iPad. The only real need that I have for the laptop is for import of my photos from my Nikon D40. I’ve begun to question whether I really need the laptop solely for immediate import of my photos. My Aperture 3 library is synced across my backup server, iMac, and laptop. I’ve begun to think that as long as I can wait to import my photos from a weekend out of the city till I get back home, I have no justifiable need for my laptop. I will be continuing to evaluate this over the next month or so, but I’m afraid that the laptop will be going. Never thought I’d say this.

I can extoll the various features of the iPad. However, I feel enough other reviews have done that. This piece is about how the iPad has changed the way I’m using technology. The next posts that I write will likely be shaped from an iPad user’s experience.

Related posts:

1. Apple iPad
2. Microsoft vs. iPad
3. iPad Tethering

What is Flash used for? In my experience, Flash is generally used for the following:

• Ads
• Games
• Music Players
• Movie Players
• Photo Galleries

Lets look at each one of these categories in a bit more detail.

Ads: Those annoying, flashy (excuse the pun) banners you see on some of the riff-raff sites of the internet. The big, ugly, strong-coloured patches in the websites you visit… generally are Flash. Very few people appreciate Flash ads. As a matter of fact, they were one of the leading instigators in the ad-blocking plugin market.

Games: This I believe is the only category that truly justifies the ‘interactive’ benefit that Flash provides. There is nothing else that remotely comes close to the power of Flash interactivity for web interface games. However, I have a problem with the category. It is generally either children or reclusive adults that tend to play Flash games. The majority of the adult population use the web for information consumption. The majority of children/teenagers will use a dedicated gaming device. I also think that as the iTunes App Store market matures in the gaming category, the majority of users will find much more value in natively coded games.

Music Streaming Another category that Flash is very prevalent in usage. Sites like MySpace, Last.fm, and others tend to use a Flash player by default. The reason for this is that Flash tends to be a very quick, easy to implement solution for music playback and playlist generation. There is nothing else available right now that offers the webmaster anything nearly as easy to implement. Until HTML5 becomes a bit more prevalent, Flash will be used de facto for music playback in the browser. Fortunately for us iPhone/iPod Touch/iPad users, some of these sites have published API’s, and some have their own dedicated app in the iTunes App Store (e.g., Last.fm). Also, the majority of iPod Touch/iPhone/iPad users will have iTunes syncing their music to their device.

Movie Players I would argue that this category would come close to Music Streaming in prevalence of Flash implementation. Unfortunately for Adobe, HTML5 offers a native implementation for movie streaming, via the ‘video’ tag. As HTML5 browsers become more prevalent, and sites like YouTube switch to either a single HTML5 portal, or a dual HTML5/Flash service, Adobe will have a harder time justifying the need for Flash in the Movie Player category.

Photo Galleries As an amateur photographer, I’ve found that the many self-hosted photo gallery solutions for servers are Flash based. However, this is again generally due to laziness on the part of programmers, or because there are a lot of easy, of-the-shelf solutions for the photographers with little computer knowledge. There are many native web coded galleries that work just as well, and are not hard to implement. Unfortunately, the majority of photographers do not have much of a budget for displaying their photos on the web, and generally hire a low-cost (usually inexperienced) web developer to build a photo site for them. There’s no technical justification for using Flash to display a gallery of photos.

Over the next few years, as HTML5 capable browsers become more ubiquitous, and providers start providing dual-service HTML5/Flash streaming, the need for Flash Movie Players will become miniscule. Thanks to the iTunes App Store, the desire for Flash games is swiftly disappearing. As applications like Lightroom and Aperture continue integrating web-publishing, photographers using professional tools for their photos will have less and less need to hire a third-party, inexperience web developer for Photo Galleries. In the end, the only justification for a webmaster to use Adobe Flash is for publishing Ads, and using it as an easy-to-implement Music Streaming player.

Related posts:

1. Response to John Nack & The Flash Blog
2. iTunes 9
3. Microsoft vs. iPad

While an OS itself can be secure as a rock, there is always a human factor involved. Even the most intelligent individual makes foolish decisions and actions at times. It is very difficult to protect the user from him/herself. It is very difficult to explain that some emails requesting their information are legitimate, and others are not. Or that some website popups only look like an application window, and are not to be clicked on.

These days, attackers find information more valuable than random destruction and wreaking havoc among systems. Consequently, the majority of attacks are designed to infiltrate and recover information, while doing their best not to alert the user. After all, why turn off the information hose as long as it’s producing information?

Consequently, one of our primary concerns should be the interaction the user makes with their computing device. If it is not possible to easily educate them on legitimate activities or requests vs. illegitimate ones, make it easy for them to understand how to control what is happening. For instance, even when they provide an administrator password to a seemingly legitimate prompt, there should be no lasting, hard to control activity. For example, an application editing the Windows Registry should not be permitted to alter settings outside the application scope (sandboxing). An educated user is a safer user. We need to do more to educate our users on the safe usage of computers, and teach them how to discern legitimate activity from illegitimate. I also believe that the OS has a place in educating the user to an extent, and providing them with easy to understand control over changes they’ve made/approved.

Windows 7 has improved the control over this particular example, but unfortunately at the expense of user understanding and interaction. Linux, Unix, and OS X on the other hand, sandbox applications and their configurations, requiring minimal administrator level permissions. Consequently, the user has to very explicitly and conscientiously make a decision whenever an application requires an administrator permission level. The controls to manage any permanent changes are very easy to use in OS X. A user deletes an application, and it no longer is capable of executing. There are no core registry changes that the user needs to be concerned about removing. Configurations are in individual text files, specific to each application. I digress however.

I do not run antivirus on my Macs. However, I run a network firewall (LittleSnitch) that informs me of network traffic going in & out. Just because my OS provides fewer attack venues, and is fairly secure, does not protect me from accidentally/foolishly approving a change which will transmit my personal information.

Related posts:

1. On the Security of Client Platforms
2. Google Wave Robots
3. Extracting Exchange Contacts from iPhone