OpenBSD is my OS of choice when it comes to setting up a quick, simple UNIX-based virtual machine (VM). Need a virtual firewall? Use OpenBSD. Need a router? Use OpenBSD. Need a web server or an FTP server? Use OpenBSD. Need to run some network security tools? Use OpenBSD.

The problem is this: once I get an OpenBSD system up and running, it runs so well that I rarely have to go set up another one. Because there is then this length of time between installations, I always find myself forgetting the steps to take when installing an OpenBSD system. Thus, the need for this post and why I say it’s really for my benefit more than anything else. Next time I need to install OpenBSD in a VM for some reason, I can quickly come back and reference my list. (I will say that the installation of OpenBSD in recent versions has gotten much simpler than it was in the past.)

Oh, another disclaimer is probably necessary here, too: this is not to be considered some sort of “best practices” guide, so please don’t hammer the comments with stuff like “You know, you really should…”. This is just a quick and simple setup.

With those disclaimers out of the way, here’s the installation procedure. This was written for use with OpenBSD 4.6:

1. Boot from the OpenBSD installation ISO image. When prompted, choose “i” to install.
2. Press Enter for the default keyboard layout (unless you need a different layout, naturally).
3. Enter the system’s hostname in short form.
4. Enter the name of the network interface to configure. When installing on VMware Fusion 3.0.2 on my Macintosh, the default interface is em0. On VMware vSphere 4, the default interface is vic0.
5. Enter the IPv4 address or press Enter to use DHCP.
6. Enter the IPv6 address or press Enter to not assign an IPv6 address.
7. Press Enter to complete the configuration of network interfaces.
8. Press Enter not to perform any manual network configuration.
9. Enter and confirm the root password.
10. Press Enter to start sshd by default.
11. Press Enter not to start ntpd by default.
12. Enter “no” to indicate that you will not be running the X Window System.
13. Press Enter not to change the default console to com0.
14. Press Enter not to create an additional user. (I generally prefer to create an additional user after installation is complete.)
15. Press Enter to accept the default disk as the root disk. On my Mac running VMware Fusion 3.0.2, the default disk is wd0.
16. Press Enter to use the whole disk.
17. Press Enter to use auto layout of partitions on the disk. (I’m not sure what version of OpenBSD added this feature, but it is quite handy for simple installations.)
18. Press Enter to use the CD to install the sets. The CD in the VM should be mapped to the ISO image of the OpenBSD 4.6 install CD.
19. Press Enter to use the default CD (which showed up as cd0 on my system).
20. Press Enter to use the default path to the sets.
21. Remove the X Window System sets by entering “-x*” and pressing Enter.
22. Verify that the X Window System sets (xbase46.tgz, xetc46.tgz, xshare46.tgz, xfont46.tgz, and xserv46.tgz) are unselected, then press Enter to complete set selection. OpenBSD will start installing the sets.
23. Enter the timezone, such as “US/Eastern”.
24. Enter reboot to reboot your new OpenBSD VM. You should now be ready to perform final configuration of OpenBSD, such as using pkg_add to install packages or editing rc.conf.local to control what daemons are launched at startup. (Of course, those are tasks for an entirely different blog post).

That’s it. Again, this not a best practice/ideal installation. It’s just a “drop dead simple” installation in a VM for when you need to get something done quickly.

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

A Quick and Simple Guide to Building an OpenBSD VM

• OpenBSD as a Simple NAT Router
• New Folders with Quicksilver
• OpenBSD 4.1 on ESX Server 3.0.1
• OpenBSD pcn0 Driver Issue Resolved
• Remotely Exploitable Flaw in OpenBSD Found

Before I can describe NIV, I need to define a few key terms:

IV-capable bridge: This is a switch that understands and is aware of interface virtualization (IV). Examples of IV-capable bridges include the Nexus 5000 and the UCS 6100XP fabric interconnects.

Interface virtualizer: An interface virtualizer (IV) is a device that simply extends the reach of an IV-capable bridge. To the outside world, an IV-capable bridge and all of its IVs appear as a single bridge. Examples of IVs include the Nexus 2000 fabric extender and the I/O module (IOM) in the Cisco Unified Computing System (UCS). There are also other forms of IVs, as I’ll explain later in this post.

Link-local tag: Frames entering an IV from a network interface card (NIC) have a link-local tag added to them; this link-local tag denotes the source IV port. Similarly, frames entering an IV-capable bridge have a link-local tag added to them that indicates the path through the IV(s) to the destination IV port. In this way, the link-local tag supplants the MAC address as the primary method of determining to which port (or out which port) a frame should be forwarded. The link-local tag is removed from the frame when it exits an IV (headed into a NIC) or when it exits an IV-capable bridge. (This link-local tag is what Cisco refers to as VNTag.)

Now that I’ve defined some NIV terminology, I’d like to explain why NIV is useful. To understand the value of NIV, take a look at the progression or development of data center networking:

1. Before virtualization, bridges (or switches) were connected to a NIC (sometimes multiple NICs, but usually just one NIC) in a physical server. The relationship between physical NICs, physical switch ports, and MAC addresses is static and easy to determine (generally one NIC with one MAC address connected to one switch port). The bridge hierarchy is reasonably simple.
2. Then hardware manufacturers introduce the server blade form factor. This introduces another layer of bridges (switches) in the blade chassis themselves and creates a more complex bridge hierarchy. This more complex bridge hierarchy also means more management, as each of these bridges represents a point of management. In order to provide redundancy between the various layers of bridges multiple connections are necessary; this, in turn, necessitates the use of Spanning Tree Protocol (STP) in order to prevent bridging loops. STP creates active/passive connections, meaning the effective bandwidth of the network is reduced.
3. Along comes virtualization, which introduces the idea of multiple virtual NICs (vNICs) associated with a single physical NIC. This, in turn, introduces another layer of switching and more points of management, and the bridge hierarchy grows more complex. Because multiple MAC addresses are now associated with a single NIC connected to a single switch port, bridges must now deal with “hairpin forwarding,” in which the switch must retransmit a frame out the same port on which it was received. The relationship between NICs, MAC addresses, and switch ports is now much more complex and very dynamic (due to live migration technologies).

As the proliferation of virtualization continues, this trend toward increased complexity also continues unabated. How, then, are we supposed to address this? NIV is intended to help address this problem. NIV seeks to remove the complexity from the edge—the NICs and vNICs—and drive that complexity toward the bridges. That is a key underlying principle behind NIV. Look back at the definitions: one characteristic of an IV-capable bridge is that the IV-capable bridge and all of its associated IVs appear to the outside world as a single bridge.

For example, consider a Nexus 2000 fabric extender connected to a Nexus 5000 switch. From both a management perspective as well as a networking perspective, the Nexus 5000+Nexus 2000 combination appears as a single device. The Nexus 2000 is simply an extension of the Nexus 5000 (hence the name “fabric extender”). This is why we say that a Nexus 2000 is an example of a interface virtualizer and why a Nexus 5000 is provided as an example of an IV-capable bridge. Similarly, this is why an IOM in the back of a UCS blade chassis functions as an interface virtualizer; it acts as an integrated part of the UCS 6100XP fabric interconnect. Just as the ports on a Nexus 2000 appear to be part of the Nexus 5000, the ports on a UCS IOM appear to be part of the UCS 6100XP fabric interconnects.

By now it should start making a little bit of sense. IVs allow you to scale to larger port densities, but they keep the complexity away from the edge of the network. If you’re astute, though, you’ll note that the examples I’ve provided so far don’t address the growth of vNICs created by virtualization. The Nexus 2000 and the UCS IOM help address the growth of physical ports, but not virtual ports. Can NIV help address vNICs as well?

So far, the examples I’ve provided of IVs have been IVs that embrace the bridge/switch form factor. There’s no reason, though, that an IV must look or feel like existing bridges or switches. Here’s another type of interface virtualizer that will really blow you away: what about the Cisco Virtual Interface Controller (VIC), aka Palo? Think about it: a VIC is really just an IV built into a UCS server blade. It appears to the outside world as additional ports on the IV-capable bridge (the UCS 6100XP fabric interconnect) to which it is connected. This underscores the flexibility of IVs and also underscores the fact that IVs can be “chained,” just as the VIC (an IV built onto the server blade) is “chained” behind the UCS IOM (an IV in the rear of the UCS chassis). Chained IVs appear as part of the upstream IV-capable bridge to which they are connected.

By building the IV into the server itself and leveraging hypervisor bypass (VMDirectPath in VMware vSphere), Cisco can address the growth of vNICs. With VIC as an IV on a VMware vSphere host, the ports that connect directly to a VM appear as ports on the upstream IV-capable bridge (the UCS 6100XP in this case); this erases any distinction between physical NICs on physical servers and virtual NICs on virtual servers. Again, a key component of NIV is that an IV-capable bridge and all associated IVs (regardless of form factor) appear as a single bridge.

Related to NIV is the idea of an Ethernet Host Virtualizer (EHV). EHV describes the behavior when an IV-capable bridge and associated IVs appear to the rest of the network as a single host. Because it appears as a single host, all issues with multiple uplinks and STP now go away. This is why, for example, the uplinks on the UCS 6100XP fabric interconnects are always active-active. I’m planning on delving a bit deeper in EHV in the near future.

I hope that this discussion of NIV has been useful. If you are interested in some additional information, I found some documents which were extremely helpful in solidifying my information. These documents (here, here, and here) are very technical but they are good sources of information nevertheless.

In a future post, I’ll discuss the role of the Nexus 1000V in NIV and how it relates to the other components described here.

Feel free to post any questions, comments, or clarifications below.

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

Understanding Network Interface Virtualization

• Understanding NPIV and NPV
• Connecting Nexus 5000 to Older Gigabit Ethernet Switches
• Identifying ESX Server NICs in Blades
• No Such Thing as an End-to-End FCoE Solution
• Why No Multi-Hop FCoE?

Cannot create a quiesced snapshot because the create snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine.

As it turns out, the problem was actually VSS in the Windows Server 2003-based guest. Since RM leverages VSS, an error with VSS was causing the entire process to fail. The fix was to clean up VSS as described in this Microsoft KB article and then reinstall the VMware Tools. After completing both of those steps, the problem was resolved.

If you are using RM and run into this problem, be sure to double-check to ensure that VSS is working as expected.

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

VSS Can Cause Problems with RM

• Snapshot Issue with VMware Data Recovery
• Partner Exchange 2010 Session TECHBC0320
• VM File-Level Recovery with NetApp Snapshots
• Recovering Data Inside VMs Using NetApp Snapshots
• Lessons Learned About Exchange Server 2007

• You might recall that in early 2008 I wrote about how thin provisioned VMDKs on NFS storage tend to inflate. In a recent post, Chad Sakac pointed out that VMware has addressed this problem, which is caused by the use of the eagerzeroedthick VMDK format instead of the zeroedthick format. The fix requires ESX 3.5 Update 5 and VirtualCenter 2.5 Update 6, plus a configuration change that is outlined in this VMware KB article. Kudos to VMware for fixing the underlying issue instead of just forcing customers to upgrade to vSphere.
• VMware’s Scott Drummonds provides a bit more information on the memory compression technology previewed by Steve Herrod at Partner Exchange 2010 a few weeks ago. In my opinion, anyone who says that the hypervisor is a commodity isn’t paying attention to the fact that VMware is still innovating in this space.
• If you perform virtualization assessments using VMware’s Capacity Planner tool, you’ll find Gabe’s Capacity Planner troubleshooting tips helpful.
• Gabe also published a “wish list” for VMware datastores. If you take a deeper look at what Gabe is really trying to address, though, a great deal of the functionality he’s looking for could be achieved through a combination of policy-based storage tiering and greater integration between VMware and the storage array. Would you really need category labels on VMware datastores if the underlying storage was tiering data effectively based on utilization? Probably not. It might still make sense in some cases, but I think the vast majority of cases would be addressed. I think that you are going to see some very cool innovation in this space over the course of this year.
• Simon Gallagher recently asked this question: with the move to ESXi, is NFS more useful than VMFS? It appears that a large part of Simon’s argument centers around the speed at which files can be transferred into VMFS using ESXi, and it seems to me that VMware needs to do some optimization there. I’m not knocking NFS—I’ve used it extensively in the past and I have and continue to recommend it to customers where it is appropriate—but I’m not sure that you can build an argument for NFS based on ESXi’s file transfer performance. My friend and former colleague Aaron Delp (whose blog was recently added to Planet V12n; congrats!) points out that fixing VMDK alignment using ESXi could be an issue; now that’s a great point. Even third-party utilities like vOptimizer don’t work with ESXi. In my opinion, these points underscore the need for VMware to concentrate very heavily on ESXi if that is indeed going to be the “platform moving forward”.
• I came across an interesting VMware KB article while browsing the weekly VMware KB digest for the week ending 2/28/10. The article, which discusses a situation in which VMware HA would fail to configure at 90% completion, describes how some network switches—HP ProCurve 1810G switches with automatic denial-of-service protection enabled and Cisco Catalyst 4948 switches with ICMP rate limiting enabled—can drop packets that are necessary for VMware HA to configure and start correctly.
• Unfortunately, the latest VMware KB weekly digest (for the week ending 3/6/2010) didn’t include links to the actual articles that were published. Bummer! Still, it’s easy enough to simply look up the articles directly.
• EMC today released a couple of plug-ins for vCenter Server. The Celerra plug-in for VMware Environments brings Celerra NFS provisioning into the vSphere Client. The Celerra Failback Plug-in for SRM automates failback in VMware SRM environments. The official press release is here, which contains links to more information on the individual plug-ins. (Disclaimer: I work for EMC.)
• Newly-minted VCDX #029 Frank Denneman posted a good article on using reservations on resource pools to bypass slot sizing. As Frank points out, it’s not a recommended practice necessarily, but it might be warranted depending on customer requirements.
• Duncan’s recent article on the behavior of CPU and memory reservations is also helpful, especially for those who might not be familiar with the differences between the two types of reservations.
• Similarly, this guest post on Duncan’s site by VCDX Craig Risinger also helps explain how shares on a resource pool work. This is good information to have if you are unfamiliar with the topic.
• I’m not a security geek, but I did think that the RSA-Intel-VMware announcement at RSAC 2010 (third-party coverage here) was pretty cool. Security experts, I’d love to hear your thoughts on the matter. What was good about the announcement? What was missing?
• If you will be working with distributed vSwitches, this post by EMC’s Gregg Robertson might help; it underscores the need to ensure that your environment is being consistently and thoroughly patched and maintained. vCenter Update Manager, anyone?

I do have a few other articles in my “things to read list” that I haven’t yet gotten around to reading:

The Official Quest Software Desktop Virtualization Group Blog » Blog Archive » How to Integrate ThinApp with Quest vWorkspace 7.0
DRS Resource Distribution Chart
HP Flex-10 versus Nexus 5000 & Nexus 1000V with 10GE passthrough

That’s it for now. I hope that you’ve found something useful here, and—as always—I’d love to hear your thoughts in the comments below.

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

Virtualization Short Take #36

• Virtualization Short Take #30
• Virtualization Short Take #35
• Virtualization Short Take #31
• Virtualization Short Take #32
• Virtualization Short Take #33

“VMware ESX/ESXi support both Fibre Channel and iSCSI storage. However, VMware and EMC do not support connecting VMware ESX/ESXi servers to CLARiiON Fibre Channel and iSCSI devices on the same array simultaneously.”

What? No Fibre Channel and iSCSI from the same array to a VMware ESX/ESXi host simultaneously? That piqued my curiosity, so I contacted a few people within EMC to question the veracity of that statement. It turns out that the answer is more complicated than it might seem at first glance.

For those of you who aren’t interested in the deep technical details, here’s the short explanation behind this behavior:

• VMware fully supports the use of both Fibre Channel and iSCSI from the same array to the same VMware ESX/ESXi host simultaneously.
• VMware does not support presenting the same LUN via both protocols concurrently to the same host. (I qualified this directly with VMware.)
• For a Celerra, you can use both Fibre Channel (via the CLARiiON side of the array) and iSCSI (via the Celerra side of the array) simultaneously. This is a fully supported configuration.
• A CLARiiON array can easily present the same LUN via both Fibre Channel and iSCSI, but then VMware wouldn’t support it (see earlier bullet).
• With a CLARiiON array, it is possible to present some LUNs via Fibre Channel and some LUNs via iSCSI to the same VMware ESX/ESXi host (i.e., LUN A via Fibre Channel and LUN B via iSCSI), but EMC will only support it if you file an RPQ. Without an RPQ, it’s an unsupported configuration. An RPQ, by the way, is a request to qualify a certain configuration for support.

I’m confident that some other array vendors out there will be very quick to jump on this post and harp on this limitation until the cows come home. I would just ask this question: is it really as big of a limitation as it seems? I’ll come back to that question in a moment.

With the short explanation in mind, here are the more in-depth details. If you like the longer, more technical explanation, then read on!

From EMC’s side, the root of the restriction about using both Fibre Channel and iSCSI devices on the same array simultaneously stems from the interaction of host registration and storage groups.

Host registration is a requirement in the CLARiiON world. In order to present storage to a host from a CLARiiON array, you must first register the host’s initiators with the array in Navisphere. Once the host has been registered, then you can proceed with presenting storage to that host. In theory the CLARiiON could operate without registering hosts and initiators, but EMC chose to require registration. EMC made this choice in order to help simplify host management.

Requiring host registration is a bit different than some of other storage arrays on the market. It’s not better or worse—just different. (Remember, pros and cons come from every technology decision.)

If you’re like me, you’re probably wondering at this point how requiring host registration simplifies anything. Instead of having to manage multiple paths, multiple initiators, and individual hosts every time you want to present storage to a host, you only need to register the host—and all of its initiators—and then you can refer to that same object (the host) over and over again as needed. Yes, host registration does mean a bit more work up front, but the idea is that it will save some work down the road. I guess you can think of host registration kind of like defining aliases in your Fibre Channel zoning configuration: it’s a bit more work up front, but it simplifies things later down the road. If you didn’t create device aliases in your Fibre Channel switch, you’d end up having to re-enter Fibre Channel WWPNs multiple times. You create the aliases so that it’s easier later. The same applies to host registration. Again, it’s a matter of choices.

One might also say that registration is security measure, albeit a weak measure. Rather than allow just any Fibre Channel-attached or iSCSI-attached host to see storage, the array requires that it know about the host (via host registration) in order to present storage to the host. This provides an additional layer of security to ensure that only authorized hosts are presented storage from the array.

Now you have a fairly decent idea of why host registration is necessary. So how does host registration occur? Host registration can occur either manually or automatically. Starting with version 4.0, both VMware ESX and VMware ESXi will automatically register with a CLARiiON array running any recent version of FLARE (ESX 3i version 3.5 also supports this form of push registration). FLARE release 28 and earlier will show these hosts as “Manually registered, unmanaged”; starting with FLARE 29, these hosts are listed as “Manually registered, managed”. In either case, the registration occurs automatically. If the host is Fibre Channel-attached, then the Fibre Channel initiators will be included in the automatic registration. The same goes for iSCSI initiators. Normally, this is a good thing because it saves the administrator the extra steps of registering the host with the storage array. (Also, because VMware ESX/ESXi hosts register automatically, there is no need to install the Navisphere Agent.)

In this case, though, the automatic registration causes a problem. Why? This goes back to the second item I said I needed to discuss: storage groups. Specifically, storage groups have two characteristics that come into play here:

1. First, any given host—not just VMware ESX/ESXi hosts, but all types of hosts—can only be connected to a single storage group at any given time.
2. Second, while the CLARiiON can present Fibre Channel LUNs and iSCSI LUNs simultaneously (including presenting the same LUN via both protocols simultaneously), there is no way within a single storage group to specify which LUNs should be accessed via Fibre Channel and which LUNs should be accessed via iSCSI. This is necessary because VMware won’t support accessing the same LUN via both protocols at the same time (see earlier VMware support statement).

Do you see how all the pieces come together? The only way to control which LUNs should be presented via which protocol is to use multiple storage groups—but a host can only be in a single storage group at a time. With only a single host object for any given VMware ESX/ESXi host, that host can only see either Fibre Channel LUNs (by being in a storage group containing Fibre Channel LUNs) or iSCSI LUNs (by being in a storage group containing iSCSI LUNs), but not both. Hence, the statement in the CLARiiON document I referenced in the very beginning of this blog post that outlines using either Fibre Channel or iSCSI but not both. This behavior is required to enforce the single-protocol LUN access required by VMware.

As with all things, there is a workaround. Because it is a workaround, that’s why the RPQ is necessary to get full support.

To work around this problem, you’ll need to ignore the automatic host registration (or disable the automatic host registration) and instead create two manually registered “pseudo-hosts”: one with the Fibre Channel initiators and one with the iSCSI initiators. These “pseudo-hosts” will need fake IP addresses (if they both use the same IP address, Navisphere will treat them as the same host, thus defeating the purpose of the workaround). Put the Fibre Channel initiators into the Fibre Channel storage group(s), and put the iSCSI initiators into the iSCSI storage group(s). Each “pseudo-host” will be able to see LUNs presented to that storage group and therefore would see both Fibre Channel and iSCSI LUNs at the same time. And, as required by VMware, any given LUN would be accessed only via Fibre Channel or iSCSI but not both. Remember that you need to file an RPQ in order to get support on this configuration.

For VMware ESX/ESXi 4.0 hosts (and ESX 3i version 3.5 hosts), you can disable automatic registration using the Disk.EnableNaviReg advanced configuration option. Setting this value to 0 disables the automatic registration with Navisphere. (Here are screenshots for VMware ESX 3i and VMware ESX/ESXi 4.) If you disable the automatic registration, then you only need to manually register the Fibre Channel and iSCSI initiators as separate “pseudo-hosts” and you’re ready to go.

Let me reiterate again that if you are presenting iSCSI LUNs via the Celerra and not the CLARiiON, none of this applies. Presenting Fibre Channel LUNs via the CLARiiON and iSCSI LUNs via the Celerra to the same VMware ESX/ESXi host is fine. This workaround that I’ve described only applies when you want to present some LUNs via Fibre Channel and some LUNs via iSCSI from a CLARiiON to a single VMware ESX/ESXi host.

Earlier you’ll recall that I asked this question: is this really a limitation? There are a couple of viewpoints:

• One viewpoint states there is no need for both Fibre Channel and iSCSI connectivity to the same array. Since you already have Fibre Channel connectivity to the array, what’s the point in using iSCSI? Conversely, if you already have iSCSI connectivity to an array, why invest in establishing Fibre Channel connectivity? Since you can’t use it for failover (that would violate the VMware support position), running another block protocol against the same array and same sets of disks doesn’t add a great deal of value.
• A second viewpoint argues that the ability to provide a differentiation of service based on the different performance characteristics of Fibre Channel and iSCSI (and NFS, but we’re focusing on block protocols for this discussion) is valuable, and thus the need to be able to easily present LUNs via either protocol from the same array to the same host is a worthwhile function. There are a number of potential use cases here—test/development environments, Tier 2 applications, varying SLAs, etc. This is especially true if you are using different disk pools (fast Fibre Channel drives or EFDs vs. slower SATA drives) on the same array.

I can see both sides of the coin. Personally, I tend to side more with the second viewpoint and would prefer to see the CLARiiON have the ability to easily present Fibre Channel and iSCSI to the same host, especially when multiple disk pools are involved. I think that CLARiiON engineering is now evaluating this possibility; as more information emerges, I’ll be sure to keep you posted.

Courteous and professional comments, clarifications, or corrections are always welcome!

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

VMware ESX, EMC CLARiiON Arrays, and Multiple Protocols

• NetApp iGroup Strategies for VMware ESX
• Sanrad Configuration Basics
• 4198: Hyper-V on NetApp Deep Dive
• iSCSI from NetApp to ESX Server
• Understanding NPIV and NPV

Before I get into the details, I’ll provide this disclaimer: there are probably easier ways of making this work. I specifically didn’t use UDA or similar because I wanted to gain the experience of how to do this the “old fashioned” way. I also wanted to be able to walk the customer through the “old fashioned” way and explain all the various components.

With that in mind, here are the components you’ll need to make this work:

1. You’ll need a DHCP server to pass down the PXE boot information. In this particular instance, I used an existing Windows-based DHCP server. Any DHCP server should work; feel free to use the Linux ISC DHCP server if you prefer.
2. You’ll need an FTP server to host the kickstart script and VMware ESX 4.0 Update 1 installation files. In this case, I used a third-party FTP server running on the same Windows-based server as DHCP. Again, feel free to use a Linux-based FTP server if you prefer.
3. You will need a TFTP server to provide the boot files. The third-party FTP server used in the previous step also provided TFTP functionality. Use whatever TFTP server you prefer.

Make sure that each of these components is working as expected before proceeding. Otherwise, you’ll spend time troubleshooting problems that aren’t immediately apparent.

Preparing for the Automated ESX Installation

First, copy the contents for the VMware ESX 4.0 Update 1 DVD—not the actual ISO, but the contents of the ISO—to a directory on the FTP server. Test it to make sure that the files can be accessed via an anonymous FTP user.

Also go ahead and create a simple kickstart script that automates the installation of VMware ESX. I won’t bother to go into detail on this step here; it’s been quite adequately documented elsewhere. You’ll need to put this kickstart script on the FTP server as well.

At this point, you’re ready to proceed with gathering the PXE boot files.

Gathering the PXE Boot Files

The first task you’ll need to complete is gathering the necessary files for a PXE boot environment.

First, copy the vmlinuz and initrd.img files from the VMware ESX 4.0 Update 1 ISO image. Since I use a Mac, for me this was a simple case of mounting the ISO image and copying out the files I needed. Linux or Windows users, it might be a bit more complicated for you. These files, by the way, are in the ISOLINUX folder on the DVD image.

Next, you’ll need the PXE boot files. Specifically, you’ll need the menu.c32 and pxelinux.0 files. These files are not on the DVD ISO image; you’ll have to download Syslinux from this web site. Once you download Syslinux, extract the files into a temporary directory. You’ll find menu.c32 in the com32/menu folder; you’ll find pxelinux.0 in the core folder. Copy both of these files, along with vmlinuz and initrd.img, into the root directory of the TFTP server. (If you don’t know the root directory of the TFTP server, double-check its configuration.)

You’re now ready to configure the PXE boot process.

Configuring the PXE Boot Environment

Once the necessary files have been placed into the root directory of the TFTP server, you’re ready to configure the PXE boot environment. To do this, you’ll need to create a PXE configuration file on the TFTP server.

The file should be placed into a folder named pxelinux.cfg under the root of the TFTP server. The filename of the PXE configuration file should be named something like this:

01-<MAC address of network interface on host>

If the MAC address of the host was 01:02:03:04:05:06, the name of the text file in the pxelinux.cfg folder on the TFTP server would be:

01-01-02-03-04-05-06

The PoC in which I was engaged involved Cisco UCS, so we knew in advance what the MAC addresses were going to be (the MAC address is assigned in the UCS service profile).

The contents of this file should look something like this (lines have been wrapped here for readability and are marked by backslashes; don’t insert any line breaks in the actual file):

default menu.c32
menu title Custom PXE Boot Menu Title
timeout 30
 
label scripted
menu label Scripted installation
kernel vmlinuz
append initrd=initrd.img mem=512M ksdevice=vmnic0 \
  ks=ftp://A.B.C.D/ks.cfg
IPAPPEND 1

You’ll want to replace ftp://A.B.C.D/ks.cfg with the correct IP address and path for the kickstart script on the FTP server.

Only one step remains: configuring the DHCP server.

Configuring the DHCP Server for PXE Boot

As I mentioned earlier, I used the Windows DHCP server as a matter of ease and convenience; feel free to use whatever DHCP server best suits your needs. There are only two options that are necessary for PXE boot:

066 Boot Server Host Name (specify the IP address of the TFTP server)
067 Bootfile Name (specify pxelinux.0)

In this particular example, I created reservations for each MAC address. Because the values were the same for all reservations, I used server-wide DHCP options, but you could use reservation-specific DHCP options if you wanted different boot options on a per-MAC address (i.e., per-reservation) basis.

The End Result

Recall that this PoC was using Cisco UCS blades. Thus, in this environment, to prepare for a new host coming online we only had to make sure that we had a PXE configuration file and create a matching DHCP reservation. The MAC address would get assigned via the service profile, and when the blade booted then it would automatically proceed with an unattended installation. Combined with Host Profiles in VMware vCenter, this took the process of bringing new ESX/ESXi hosts online down to mere minutes. A definite win for any customer!

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

PXE Booting VMware ESX 4.0

• Client-Specific DHCP Options with Linux DHCP Server
• Modifying DHCP Client Operation on OpenBSD
• Badmail and Exchange 2003
• Full VM Recovery with NetApp Snapshots
• Apparent Date/Time Issue With Update 2

The first big piece of news was that HyTrust hired Jim Gannon, a former VMware executive, to serve as the VP of Sales. The full press release for that announcement is here.

The second big piece of news comes in the form of this press release announcing that HyTrust has secured $10.5 million in Series B financing, including an investment from Cisco Systems.

The third and final piece of news, and the one that I personally find most exciting, is that HyTrust has been named one of ten finalists for the “Most Innovative Company at RSA Conference 2010″ award. Congratulations on all three counts!

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

Congrats to HyTrust

• HyTrust Appliance Community Edition
• HyTrust Launches Security Appliance
• HyTrust Appliance 1.5
• Congrats to Hyper9
• BlueStripe Extends FactFinder to Red Hat Linux

One task that I’m finding as a member of EMC’s vSpecialist team is that there is a lot of reading. We’re responsible for reading all sorts of documents. I don’t mind doing this in the evenings, when I’m not writing for one of my upcoming books or studying for a certification exam, but I’d really much rather prefer to do this in a way that makes it possible for me to be with my family. So, having some sort of device that would allow me to review documents while I’m sitting in the den with the kids would be great.

My thought is that I could leverage something like Dropbox to synchronize documents between my MacBook Pro and an iPad. With the documents easily accessible on (or from) the iPad, I could sit on the couch and read or review documents while the kids sit next to me and watch TV or read a book. This would help me stay on top of the document reviewing without pulling me into my office and away from the family.

What do you think? Good idea, or not? Anyone else have any uses for the iPad that you’d like to describe? Speak up in the comments.

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

A Potential Use for the iPad

• vSphere 4.0 Quick Start Guide
• Welcome
• The Future of the OS
• I Tried to Like Postbox, I Really Did
• New Folders with Quicksilver

After many long hours of preparation on the application and the submitted design, after days and weeks of waiting, after hours spent reviewing the design, and after reading numerous tips and tricks from established VCDXs, it all culminated in the defense panel. There, in the defense panel, I would have to stand before three knowledgeable, established design experts and defend the choices made in the design. I’d have to explain why I chose block storage over NFS, why Fibre Channel over iSCSI, why blade servers vs. rack-mount servers. I’d have to explain why I chose the LUN size I chose, and I’d have to defend the zoning that controlled the presentation of those LUNs. Clusters, cluster sizes, features enabled or not enabled, networking layout, VM density on the LUNs, projected IOPS—nothing was safe from their inquisition. And yes, I’d have to explain why the design used NetApp storage instead of EMC storage. (It was a customer requirement, i.e., a design constraint.)

Surprisingly, when the time for the VCDX defense panel arrived I found myself a lot more nervous than I had expected I would be. After all, this was just a friendly conversation with technical peers, right? In many ways it could be viewed that way, but the underlying purpose behind the conversation was ever-present: there was a reason I was there standing before these three people. It wasn’t just “shooting the breeze” with friends; there was a purpose there. It wasn’t just bouncing ideas off co-workers or industry colleagues; there was a reason for the conversation. It’s not that the panelists did anything to cause this feeling; they were completely fine, very courteous and quite friendly. (In case you’re wondering, I’m not going to disclose who was on my panel. They are welcome to disclose if they so choose, but that will be their decision.)

Looking back on it now, I realize that I should have gotten better control of my nerves. I spoke too quickly. I rushed through questions that probably deserved more explanation. I forgot details about my design. I got tripped up by relatively simple questions. I’ve made no secret of the fact that I wasn’t pleased by how well I performed—or didn’t perform—in the VCDX defense panel. I was upset that I had been thrown off and that I wasn’t able to recall all the details from my design. For a few hours after completing the defense panel, I beat myself up over how things had gone. But it didn’t take me too long to make peace with not having passed. I knew that if I had not passed, the experience was still worthwhile as a learning experience. Even if I hadn’t gained the VCDX certification, I’d still gained knowledge and experience. And hey, there was always another chance to defend at VMworld, right?

After returning home from Las Vegas, I spent the week thinking about what I would write after I’d finally gotten my results. I tried to prepare for the questions like “How in the world could Scott not pass?” I thought about explaining that the defense panel was only doing their job; they were preserving the value of the certification. After all, if the bar is not held high, what is the value of VCDX? All the while I secretly hoped that the result would be something other than what I was confident it would be.

And so it was that as I was driving my kids to a church youth group function tonight—after a long and unproductive day working with some rather stubborn equipment in the lab—that I received an e-mail from VMware. The first line of the message was this:

Congratulations! You have achieved the VCDX3 certification. Your VCDX number is: VCDX39

Unbelievable! I’d passed! I was so excited. I’d hoped for this result, but I honestly did not believe that I had managed to pull it off. I immediately called Crystal to tell her the news. I think she might have been even more excited than me.

Having now been through this entire process, what advice do I have for aspiring VCDX candidates?

• As many others have stated, know your design. If I had only one thing to change about my entire process, this would be it. You should know it forward and backward: every detail, every choice, and every reason behind the design.
• Don’t be too nervous. I allowed my nerves to get the best of me, there’s no question. I also don’t doubt that I would have done better had I not been so nervous. (As a side note, it’s interesting to me that I can stand up and speak in front of large crowds and not be nervous, but standing in front of those panelists really threw me. Odd.)
• Understand the impact of your choices. As Duncan pointed out in this recent blog post, it’s really about the impact. Be prepared to discuss the reasons for the decisions in your design and the impact of the decisions in your design.

That’s it from the latest VCDX to join the ranks. I’ll post another update later with more tips and tricks that I learned from the experience, but those are some that jump to my mind immediately.

Have a great weekend!

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

On the VCDX Defense

• VCDX Design Exam Post-Mortem
• TA2259 Ask the Experts at VMworld 2009
• Virtualization Short Take #27
• Partner Exchange Wrap-Up
• Submit a Question for VMworld 2009 Ask the Experts Session

Swapping UCS Blades with Local Boot Policies
Get Spidey Powers with UCS; but with Great Power comes Great Responsibility
25 ways that Cisco UCS frees you to do other things
Cisco UCS - How Many FEX Uplinks Do I Need?
UCS local disk policy + some vBlock
Cisco UCS: different workload, different configuration, same blade. Simple.
Cisco UCS Information for “Server People”
Cisco UCS vs. IBM and HP - Where are the Brains?
UCS Gotchas? and how much time does it take day to day?

Anyone else have any UCS posts that have surfaced recently? Add them in the comments below.

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

A Collection of UCS Posts

• UCS Training, Day 1 Wrap-Up
• Using IP-Based Storage with VMware vSphere on Cisco UCS
• Cisco UCS Announcements
• A Collection of Viewpoints on Cisco UCS
• UCS Class Wrap-Up