Comments on: Configuring Cisco 1000v with NAS datastores http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/ technologist defying the peter principle Fri, 09 Mar 2012 17:19:54 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: rajeev karamchedu http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/comment-page-1/#comment-18630 rajeev karamchedu Thu, 17 Sep 2009 14:04:44 +0000 http://rajeev.name/?p=501#comment-18630 Thanks to Brad and Christian's for their pointers. The missing piece was the addition of system vlans on the port profile. We only had it on one side (uplink port profile). With those corrections, we have a clean vSphere 4 with 1000v configuration. Below is the configuration we used on the VSMs and works like a charm. Next up - NX 5Ks and 7Ks. Good times. <strong>VMKernel</strong> <pre lang="bash"> port-profile vmkernel-uplink capability uplink vmware port-group switchport mode trunk switchport trunk allowed vlan 151 channel-group auto mode active no shutdown system vlan 151 state enabled port-profile vmkernel vmware port-group switchport mode access switchport access vlan 151 no shutdown system vlan 151 state enabled </pre> <strong>VM Network</strong> <pre lang="bash"> port-profile vmnetwork-uplink capability uplink vmware port-group switchport mode trunk switchport trunk allowed vlan 106,997-998 channel-group auto mode active no shutdown state enabled port-profile vmnetwork-106 vmware port-group switchport mode access switchport access vlan 106 no shutdown state enabled </pre> <strong>Service Console</strong> <pre lang="bash"> port-profile service-console-uplink capability uplink vmware port-group switchport mode trunk switchport trunk allowed vlan 105 channel-group auto mode active no shutdown system vlan 105 state enabled port-profile service-console vmware port-group switchport mode access switchport access vlan 105 no shutdown system vlan 105 state enabled </pre> Thanks to Brad and Christian’s for their pointers. The missing piece was the addition of system vlans on the port profile. We only had it on one side (uplink port profile). With those corrections, we have a clean vSphere 4 with 1000v configuration. Below is the configuration we used on the VSMs and works like a charm.

Next up – NX 5Ks and 7Ks. Good times.

VMKernel

port-profile vmkernel-uplink
  capability uplink
  vmware port-group
  switchport mode trunk
  switchport trunk allowed vlan 151
  channel-group auto mode active
  no shutdown
  system vlan 151
  state enabled
 
port-profile vmkernel
  vmware port-group
  switchport mode access
  switchport access vlan 151
  no shutdown
  system vlan 151
  state enabled

VM Network

port-profile vmnetwork-uplink
  capability uplink
  vmware port-group
  switchport mode trunk
  switchport trunk allowed vlan 106,997-998
  channel-group auto mode active
  no shutdown
  state enabled
 
port-profile vmnetwork-106
  vmware port-group
  switchport mode access
  switchport access vlan 106
  no shutdown
  state enabled

Service Console

port-profile service-console-uplink
  capability uplink
  vmware port-group
  switchport mode trunk
  switchport trunk allowed vlan 105
  channel-group auto mode active
  no shutdown
  system vlan 105
  state enabled
 
port-profile service-console
  vmware port-group
  switchport mode access
  switchport access vlan 105
  no shutdown
  system vlan 105
  state enabled
]]> By: rajeev karamchedu http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/comment-page-1/#comment-18444 rajeev karamchedu Fri, 11 Sep 2009 01:20:16 +0000 http://rajeev.name/?p=501#comment-18444 @Brad and @Christian - Thanks for your help. Brad - Just spoke to Pierre a few mins ago and he mentioned the same thing that @Christian also mentioned. To answer your question, I asked for the network configs and verified that we did not have portfast enabled. (I remember asking for it, however). Things to check 1. We know that the system vlans are enabled on the uplink port profile. We need to ensure that the system vlans are enabled on the VMkernel vDS interface port profile as well. I didn't think there was a way to check that using vCenter but Christian's example shows me the way. 2. Enable spanning tree portfast trunk on the port channels 3. We are also using LACP for link aggregation. We'll need to see if there are any config parameters that might allow us to bring these ports up any faster. Please stay tuned and muchas gracias! @Brad and @Christian – Thanks for your help. Brad – Just spoke to Pierre a few mins ago and he mentioned the same thing that @Christian also mentioned.

To answer your question, I asked for the network configs and verified that we did not have portfast enabled. (I remember asking for it, however).

Things to check

1. We know that the system vlans are enabled on the uplink port profile. We need to ensure that the system vlans are enabled on the VMkernel vDS interface port profile as well. I didn’t think there was a way to check that using vCenter but Christian’s example shows me the way.

2. Enable spanning tree portfast trunk on the port channels

3. We are also using LACP for link aggregation. We’ll need to see if there are any config parameters that might allow us to bring these ports up any faster.

Please stay tuned and muchas gracias!

]]> By: Christian Elsen http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/comment-page-1/#comment-18442 Christian Elsen Thu, 10 Sep 2009 23:40:22 +0000 http://rajeev.name/?p=501#comment-18442 Hi, there is no problem in moving the VMKernel interface used for NFS or iSCSI to the Nexus 1000V. You just need to configure the VLAN that is being used for NFS or iSCSI as a system VLAN on the uplink port-profile AND the actual port-profile used for the VMKernel interface. Here's a sample config. Let's say you will use VLAN 14 for NFS or iSCSI. <pre lang="bash"> port-profile uplink_port_profile_1 capability uplink vmware port-group switchport trunk allowed vlan 10-14,21-22 channel-group auto mode on sub-group cdp no shutdown system vlan 10-11,21-22 state enabled port-profile iSCSI_or_NFS vmware port-group switchport mode access switchport access vlan 10 no shutdown system vlan 10 state enabled </pre> Same applies for the Service Console! Hope that helps Chris Hi,

there is no problem in moving the VMKernel interface used for NFS or iSCSI to the Nexus 1000V. You just need to configure the VLAN that is being used for NFS or iSCSI as a system VLAN on the uplink port-profile AND the actual port-profile used for the VMKernel interface.

Here’s a sample config. Let’s say you will use VLAN 14 for NFS or iSCSI.

port-profile uplink_port_profile_1
  capability uplink
  vmware port-group
  switchport trunk allowed vlan 10-14,21-22
  channel-group auto mode on sub-group cdp
  no shutdown
  system vlan 10-11,21-22
  state enabled
 
port-profile iSCSI_or_NFS
  vmware port-group
  switchport mode access
  switchport access vlan 10
  no shutdown
  system vlan 10
  state enabled

Same applies for the Service Console!

Hope that helps

Chris

]]> By: Brad Hedlund http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/comment-page-1/#comment-18441 Brad Hedlund Thu, 10 Sep 2009 23:36:38 +0000 http://rajeev.name/?p=501#comment-18441 OK... Do you have 'spanning-tree portfast trunk' enabled on switchports connecting the ESX host? Thanks, Brad OK… Do you have ‘spanning-tree portfast trunk’ enabled on switchports connecting the ESX host?

Thanks,
Brad

]]> By: rajeev karamchedu http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/comment-page-1/#comment-18435 rajeev karamchedu Thu, 10 Sep 2009 22:05:58 +0000 http://rajeev.name/?p=501#comment-18435 Yes - They are system vlans. Today we moved SC and CPM (Control, Packet and Management) over to the vDS under the thinking that the vDS is going to get the SC and VMkernel available more cleanly so we can mount the NAS datastores. . That really did not help things all that much. We still see the ESX Server boot up, *hang* and NAS datastores for about 10 minutes and finally mount the datastores. During that time, we see the links come up *twice* followed by NAS datastore mounting. The VMKernel IP and the storage IPs are in same subnets and otherwise there are NO nfs errors (RPC or any other kind). It's only at the boot time we get this delay. Yes – They are system vlans. Today we moved SC and CPM (Control, Packet and Management) over to the vDS under the thinking that the vDS is going to get the SC and VMkernel available more cleanly so we can mount the NAS datastores. . That really did not help things all that much. We still see the ESX Server boot up, *hang* and NAS datastores for about 10 minutes and finally mount the datastores. During that time, we see the links come up *twice* followed by NAS datastore mounting.

The VMKernel IP and the storage IPs are in same subnets and otherwise there are NO nfs errors (RPC or any other kind). It’s only at the boot time we get this delay.

]]> By: Brad Hedlund http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/comment-page-1/#comment-18433 Brad Hedlund Thu, 10 Sep 2009 21:31:48 +0000 http://rajeev.name/?p=501#comment-18433 Oh, also, do you have 'system vlan' configured in the Port Profiles used as uplinks? And is the VMKernel's VLAN listed in this command? Thanks, Brad Oh, also, do you have ‘system vlan’ configured in the Port Profiles used as uplinks? And is the VMKernel’s VLAN listed in this command?

Thanks,
Brad

]]> By: Brad Hedlund http://rajeev.name/2009/09/09/configuring-cisco-1000v-with-nas-datastores/comment-page-1/#comment-18432 Brad Hedlund Thu, 10 Sep 2009 21:27:14 +0000 http://rajeev.name/?p=501#comment-18432 Do you the 'system vlan' command in the Port Profile in use by the VMKernel? Do you the ‘system vlan’ command in the Port Profile in use by the VMKernel?

]]>