 | Configuring Trunking Between ESXi 5 server and CISCO Switch I've been pouring over videos from various sites (Cbt nuggets/Youtube) and not once (unless my searching is off) have a found a "clear and concise" instructional showing how to get traffic from (let's say...) "vlan 3" from the switch (be it CISCO, Juniper, or otherwise) to "vlan 3" on the esxi server while communicating to the esxi server on management vlan 1 (I say "vlan 1" just for the purpose of a basic starting point).
I've basically set the port on the switch to "trunking;" I've then gone into the console on the esxi box and set all ports to "4095" for the sake of getting my bearings before trying to be more specific. Additionally, I've configured a port group on the esxi box also to no avail :-(
Anyone doing this in either a lab or production environment?
Jay |
|
 tubbynetreminds me of the danse russePremium,MVM join:2008-01-16 Chandler, AZ kudos:1 | configure static dot1q trunks. make sure the ip addys and vlan tags sit where they need for communication. getting an etherchannel is much more complex -- and not needed for this argument.
start small.
hit me up on skype if you need more help.
q. -- "...if I in my north room dance naked, grotesquely before my mirror waving my shirt round my head and singing softly to myself..." |
|
 | reply to DocLarge The basic config on the Cisco / Juniper / et al side would be your basic switchport / trunk commands you've ever learned, including but not limited to :
switch#
switchport mode trunk
switchport permited vlans 1,3
switchport native vlan x dot1q x
...
At that point it'd be up to ESXi and how it configures the VLANs and what traffic traverses across which VLAN, of which I don't claim any config or operational experience on ESXi that score.
My 00000010bits.
Regards |
|
 cramer join:2007-04-10 Raleigh, NC kudos:7 | reply to DocLarge I do this every day. I find it absolutely trivial. What are you using to manage your ESXi server(s)? VLANs cannot be setup from the console (the yellow screen). They can be setup via CLI, but I wouldn't go there. The viclient is the recommended path -- either direct to the server or through vCenter.
Create a vSwitch. Assign a NIC to the vSwitch. Create your networks on the vSwitch. Within the configuration of each network is a setting for the VLAN -- "0" is untagged. The Cisco side is the same as any other trunk port. |
|
 | reply to HELLFIRE @ hellfire: Good god, man!!! What IOS version are you using??? *heh* I looked around for the "switchport permitted" command you referenced and I didn't find it anywhere. Did you mean "switchport trunk allowed vlan add 1, 3 ?"
Jay |
|
 | reply to cramer
Cramer,
here are some attached shots of what I'm working with. I may need to map this out better... |
|
|
|
 | reply to DocLarge @cramer Thanks for that info. The next quote, unquote "server admin" that tries to claim a "network issue" when setting up ESXi in this fashion, I'm gonna ask them "did you set up your VLANs right during initial config like this, this, and this?"
@DocLarge ...I didn't have an IOS switch in front of me when I went thru those commands... but you did go thru NA and get the gist of setting up trunks in IOS, yes? 
Regards |
|
 3 edits | Yes I did, my man 
I'm still having fits right now, so I'm looking at this from another perspective. I'm actually running "router on a stick." I'll attach a shot of how I just briefly sketched things out on my whiteboard in a moment...
EDIT 1
Here's the logical layout of what I'm working. I'm amazed at how this concept is simple in conversation yet "frustrating" in action... Oh, yeah, I'm definitely the "weak link" in this scenario... 
I also found this article: »www.mustbegeek.com/virtualizatio···-server/
"Some relevance" here but the "magic bullet" I'm after (getting vlans on esxi to talk to vlans on a switch) isn't answered :-( Think I'll break out the 3550 and replace the 2924...
Edit 2
I found an old article online that speaks about esxi-to-cisco switch vlan configuration:
»kb.vmware.com/selfservice/micros···=1003806
I've already tried the links being spoken about; will probably try again... |
|
 TomS_Git-r-donePremium,MVM join:2002-07-19 London, UK kudos:4 | reply to DocLarge Not sure if Im too late, or if ESXi 5 has changed since ESXi 4 but ...
When I set up my ESXi 4 host, I (had to?) configure the management VLAN ID from the text based configuration tool that you get access to from the box physically with a keyboard and screen after it boots up.
Log in and go in to a certain configuration section and you can set the VLAN ID to use for management. It is then a simple matter of trunking that VLAN ID to the box and youre off. Personally, I never use VLAN 1 for anything these days, I always choose a different VLAN for management, so Im not sure how this goes when it comes to native VLANs...
As for your screen shot showing no observed IP ranges, I get the same thing on my ESXi 4.1 host, so probably nothing wrong there (I guess because all of my packets are encapsed with VLAN headers as well.)
If you cant get management access going with a native VLAN, could you create a VLAN to do nothing (like 666 for evil), and set that as the native VLAN, allowing VLAN 1 to be tagged over the trunk, and configure your host to look for VLAN 1 as the management VLAN?
I think you might also be able to configure the VLAN ID of the management network from within the vSphere console by editing the VLAN associated with "vMotion and IP Storage". |
|
 | No answer as of yet, Tom... You know how it goes, we all know how to do a task, but sometimes we have a bit of time trying to relay it to someone else.... I still appreciate the insight from everyone... I'm taking another break due to annoyance at this point. Seriously, I know it can't be this difficult to achieve what I'm after. *groan*
Any chance Dslreports will start a "Virtualization" forum? |
|
 TomS_Git-r-donePremium,MVM join:2002-07-19 London, UK kudos:4 | said by DocLarge:Any chance Dslreports will start a "Virtualization" forum? Maybe about the same time they open up a Juniper forum.  |
|
 | I'm sensing "snowball's chance in hell" on this one... *heh*  |
|
 cramer join:2007-04-10 Raleigh, NC kudos:7 | reply to DocLarge
 iSCSI |  |
You don't set the VLAN to 4095. That's an indicator to ESX that the VMs will be doing tagging.
If the switch port connected to the vmnic is not tagging frames, the VLAN must be zero -- if you set a number, the vswitch will be looking for that dot1q tag, which won't be there.
For "trunk" switchports, the native vlan maps to vlan 0 on ESX no matter what VLAN it is on the switch; to the vswitch, zero means not tagged. The tagged vlans then map to the network with the same vlan on the vswitch.
In both pictures, vSwitch0 networks are set to zero. There could be any VLAN on the switch side, but vmware doesn't know about it, and doesn't need to. The first vSwitch1 has a single (kernel) network for iSCSI, and it's in VLAN 4000; there may be (and are) more vlans on the link, but vmware doesn't need to know about them. And vSwitch2 is set to "all" so VMs can put whatever they need on the link -- hence the name "Replay Network" |
|
 cramer join:2007-04-10 Raleigh, NC kudos:7 | reply to DocLarge In your diagram, ESX is connected to the network via a single ACCESS link. I'll assume that vmnic (0?) is attached to vSwitch0. ALL of the networks on vSwitch0 would have a VLAN of 0 (zero) because the switch isn't tagging any packets.
If you set the switch port to trunk, and assigned vlan 4 as the native vlan, and allowed vlans 2,3,4... vSwitch0 would have networks with VLAN: 0 (which is vlan 4 on the switch; not tagged to ESX) and networks with VLANs 2 and 3 (the tagged networks.)
vSwitch1 won't work (externally) until you assign it a vmnic that's connected to something. And you cannot put the same physical nic in multiple vSwitches. In my previous picture, vSwitch2 is attached to vmnic2 which isn't connected; that vSwitch only works *on that esx host*. (there are several hosts setup like that, with no nic assigned to the vSwitch)
[As previously mentioned "All (4095)" is a special case configuration where vmware does no processing on the traffic, passing un/tagged traffic right through to the VMs. Assigning a VMKernel or Service Console network to "All" will not work.] |
|
 | Cramer, thanks for all of the input, it's greatly appreciated. This will serve as a good baseline. Normally, when I get to this level of detail, I generally have a "step by step" guide to accompany the diagram.
Can I get that from you to better associate "action" to "outcome?"
Thanks... |
|
 cramer join:2007-04-10 Raleigh, NC kudos:7 | I don't "Do" documentation... (job (ob)security)
Maybe not in a 1, 2, 3... but everything is there in the previous posts. Starting from system install, the switch port had to be setup to allow it on the network -- installed from CD/USB, it'll ask basic questions about your network setup for the management interface. (I've never paid enough attention to the installer to remember if it asks for a vlan during setup. I netboot the installer so I have to be untagged.) You could have the switchport already setup for trunking -- native vlan being for the management interface, and tagged vlans for various VMs, etc.
From there everything is done from vcenter (add host) / viclient (direct to host.) Create any additional (tagged/untagged) networks on vSwitch0. Create any additional vSwitch's necessary for your setup.
Then configure your storage systems... local, NFS, iSCSI, fibre channel... I have iSCSI in my environment, so there's another entire book of steps I have to go through to get the storage set up; most of it not within vmware.
Then give it some VMs...
(I have 90% of this automated from an installer script. No, you cannot have that script.) |
|
 | Lol @ "No you can't have the script"  |
|
 | reply to cramer Lol @ "No you can't have the script"  |
|
 TomS_Git-r-donePremium,MVM join:2002-07-19 London, UK kudos:4 | reply to cramer said by cramer:job (ob)security Or negligence, when you get hit by a bus and the new guy doesnt know what to do.  |
|
 cramer join:2007-04-10 Raleigh, NC kudos:7 | Heh. It's more a job of "we don't know what he does, but for God's sake, you want him doing it." My coworkers are very happy I'm doing what I do; even if I documented all of it, they wouldn't want to do it. |
|