dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
15079
share rss forum feed

DocLarge
Premium
join:2004-09-08
kudos:1

Configuring Trunking Between ESXi 5 server and CISCO Switch

I've been pouring over videos from various sites (Cbt nuggets/Youtube) and not once (unless my searching is off) have a found a "clear and concise" instructional showing how to get traffic from (let's say...) "vlan 3" from the switch (be it CISCO, Juniper, or otherwise) to "vlan 3" on the esxi server while communicating to the esxi server on management vlan 1 (I say "vlan 1" just for the purpose of a basic starting point).

I've basically set the port on the switch to "trunking;" I've then gone into the console on the esxi box and set all ports to "4095" for the sake of getting my bearings before trying to be more specific. Additionally, I've configured a port group on the esxi box also to no avail :-(

Anyone doing this in either a lab or production environment?

Jay



tubbynet
reminds me of the danse russe
Premium,MVM
join:2008-01-16
Chandler, AZ
kudos:1

configure static dot1q trunks.
make sure the ip addys and vlan tags sit where they need for communication. getting an etherchannel is much more complex -- and not needed for this argument.

start small.

hit me up on skype if you need more help.

q.
--
"...if I in my north room dance naked, grotesquely before my mirror waving my shirt round my head and singing softly to myself..."


HELLFIRE
Premium
join:2009-11-25
kudos:18
reply to DocLarge

The basic config on the Cisco / Juniper / et al side would be your basic switchport / trunk commands you've
ever learned, including but not limited to :

switch#
switchport mode trunk
switchport permited vlans 1,3
switchport native vlan x dot1q x
...
 

At that point it'd be up to ESXi and how it configures the VLANs and what traffic traverses across which
VLAN, of which I don't claim any config or operational experience on ESXi that score.

My 00000010bits.

Regards

cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:8
reply to DocLarge

I do this every day. I find it absolutely trivial. What are you using to manage your ESXi server(s)? VLANs cannot be setup from the console (the yellow screen). They can be setup via CLI, but I wouldn't go there. The viclient is the recommended path -- either direct to the server or through vCenter.

Create a vSwitch. Assign a NIC to the vSwitch. Create your networks on the vSwitch. Within the configuration of each network is a setting for the VLAN -- "0" is untagged. The Cisco side is the same as any other trunk port.


DocLarge
Premium
join:2004-09-08
kudos:1
reply to HELLFIRE

@ hellfire: Good god, man!!! What IOS version are you using??? *heh* I looked around for the "switchport permitted" command you referenced and I didn't find it anywhere. Did you mean "switchport trunk allowed vlan add 1, 3 ?"

Jay


DocLarge
Premium
join:2004-09-08
kudos:1
reply to cramer

Click for full size
Click for full size
Click for full size
Cramer,

here are some attached shots of what I'm working with. I may need to map this out better...

HELLFIRE
Premium
join:2009-11-25
kudos:18
reply to DocLarge

@cramer
Thanks for that info. The next quote, unquote "server admin" that tries to claim a "network issue" when setting up
ESXi in this fashion, I'm gonna ask them "did you set up your VLANs right during initial config like this, this, and this?"

@DocLarge
...I didn't have an IOS switch in front of me when I went thru those commands... but you did go thru NA and get the gist
of setting up trunks in IOS, yes?

Regards


DocLarge
Premium
join:2004-09-08
kudos:1

3 edits

Click for full size
Yes I did, my man

I'm still having fits right now, so I'm looking at this from another perspective. I'm actually running "router on a stick." I'll attach a shot of how I just briefly sketched things out on my whiteboard in a moment...

EDIT 1

Here's the logical layout of what I'm working. I'm amazed at how this concept is simple in conversation yet "frustrating" in action... Oh, yeah, I'm definitely the "weak link" in this scenario...

I also found this article: »www.mustbegeek.com/virtualizatio···-server/

"Some relevance" here but the "magic bullet" I'm after (getting vlans on esxi to talk to vlans on a switch) isn't answered :-( Think I'll break out the 3550 and replace the 2924...

Edit 2

I found an old article online that speaks about esxi-to-cisco switch vlan configuration:

»kb.vmware.com/selfservice/micros···=1003806

I've already tried the links being spoken about; will probably try again...


TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5
reply to DocLarge

Not sure if Im too late, or if ESXi 5 has changed since ESXi 4 but ...

When I set up my ESXi 4 host, I (had to?) configure the management VLAN ID from the text based configuration tool that you get access to from the box physically with a keyboard and screen after it boots up.

Log in and go in to a certain configuration section and you can set the VLAN ID to use for management. It is then a simple matter of trunking that VLAN ID to the box and youre off. Personally, I never use VLAN 1 for anything these days, I always choose a different VLAN for management, so Im not sure how this goes when it comes to native VLANs...

As for your screen shot showing no observed IP ranges, I get the same thing on my ESXi 4.1 host, so probably nothing wrong there (I guess because all of my packets are encapsed with VLAN headers as well.)

If you cant get management access going with a native VLAN, could you create a VLAN to do nothing (like 666 for evil), and set that as the native VLAN, allowing VLAN 1 to be tagged over the trunk, and configure your host to look for VLAN 1 as the management VLAN?

I think you might also be able to configure the VLAN ID of the management network from within the vSphere console by editing the VLAN associated with "vMotion and IP Storage".


DocLarge
Premium
join:2004-09-08
kudos:1

No answer as of yet, Tom... You know how it goes, we all know how to do a task, but sometimes we have a bit of time trying to relay it to someone else.... I still appreciate the insight from everyone... I'm taking another break due to annoyance at this point. Seriously, I know it can't be this difficult to achieve what I'm after. *groan*

Any chance Dslreports will start a "Virtualization" forum?



TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5

said by DocLarge:

Any chance Dslreports will start a "Virtualization" forum?

Maybe about the same time they open up a Juniper forum.

DocLarge
Premium
join:2004-09-08
kudos:1

I'm sensing "snowball's chance in hell" on this one... *heh*


cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:8
reply to DocLarge


iSCSI
You don't set the VLAN to 4095. That's an indicator to ESX that the VMs will be doing tagging.

If the switch port connected to the vmnic is not tagging frames, the VLAN must be zero -- if you set a number, the vswitch will be looking for that dot1q tag, which won't be there.

For "trunk" switchports, the native vlan maps to vlan 0 on ESX no matter what VLAN it is on the switch; to the vswitch, zero means not tagged. The tagged vlans then map to the network with the same vlan on the vswitch.

In both pictures, vSwitch0 networks are set to zero. There could be any VLAN on the switch side, but vmware doesn't know about it, and doesn't need to. The first vSwitch1 has a single (kernel) network for iSCSI, and it's in VLAN 4000; there may be (and are) more vlans on the link, but vmware doesn't need to know about them. And vSwitch2 is set to "all" so VMs can put whatever they need on the link -- hence the name "Replay Network"

cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:8
reply to DocLarge

In your diagram, ESX is connected to the network via a single ACCESS link. I'll assume that vmnic (0?) is attached to vSwitch0. ALL of the networks on vSwitch0 would have a VLAN of 0 (zero) because the switch isn't tagging any packets.

If you set the switch port to trunk, and assigned vlan 4 as the native vlan, and allowed vlans 2,3,4... vSwitch0 would have networks with VLAN: 0 (which is vlan 4 on the switch; not tagged to ESX) and networks with VLANs 2 and 3 (the tagged networks.)

vSwitch1 won't work (externally) until you assign it a vmnic that's connected to something. And you cannot put the same physical nic in multiple vSwitches. In my previous picture, vSwitch2 is attached to vmnic2 which isn't connected; that vSwitch only works *on that esx host*. (there are several hosts setup like that, with no nic assigned to the vSwitch)

[As previously mentioned "All (4095)" is a special case configuration where vmware does no processing on the traffic, passing un/tagged traffic right through to the VMs. Assigning a VMKernel or Service Console network to "All" will not work.]


DocLarge
Premium
join:2004-09-08
kudos:1

Cramer, thanks for all of the input, it's greatly appreciated. This will serve as a good baseline. Normally, when I get to this level of detail, I generally have a "step by step" guide to accompany the diagram.

Can I get that from you to better associate "action" to "outcome?"

Thanks...


cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:8

I don't "Do" documentation... (job (ob)security)

Maybe not in a 1, 2, 3... but everything is there in the previous posts. Starting from system install, the switch port had to be setup to allow it on the network -- installed from CD/USB, it'll ask basic questions about your network setup for the management interface. (I've never paid enough attention to the installer to remember if it asks for a vlan during setup. I netboot the installer so I have to be untagged.) You could have the switchport already setup for trunking -- native vlan being for the management interface, and tagged vlans for various VMs, etc.

From there everything is done from vcenter (add host) / viclient (direct to host.) Create any additional (tagged/untagged) networks on vSwitch0. Create any additional vSwitch's necessary for your setup.

Then configure your storage systems... local, NFS, iSCSI, fibre channel... I have iSCSI in my environment, so there's another entire book of steps I have to go through to get the storage set up; most of it not within vmware.

Then give it some VMs...

(I have 90% of this automated from an installer script. No, you cannot have that script.)


DocLarge
Premium
join:2004-09-08
kudos:1

Lol @ "No you can't have the script"


DocLarge
Premium
join:2004-09-08
kudos:1
reply to cramer

Lol @ "No you can't have the script"



TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5
reply to cramer

said by cramer:

job (ob)security

Or negligence, when you get hit by a bus and the new guy doesnt know what to do.

cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:8

Heh. It's more a job of "we don't know what he does, but for God's sake, you want him doing it." My coworkers are very happy I'm doing what I do; even if I documented all of it, they wouldn't want to do it.



battleop

join:2005-09-28
00000

1 recommendation

reply to TomS_

I've always told my boss that if the "new" guy can't figure out my notes and documentation then he picked the wrong "new" guy. I won't waste my time writing up SOP manuals and step by step instructions on how I do my job.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.



TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5

Haha yes. The test of a true engineer is if they can reverse engineer what already exists.

I happen to agree. Except in two cases:

•Where a specific procedure should be followed in order to achieve a certain consistent result
•Where the equipment may be so obscure that any regular engineer, no matter how good they are, may simply have not come across it

But otherwise time is too short and there are far more interesting things to be doing other than documentation (say all engineers I am sure.)

Although I will concede one thing, in my current job I handle circuit documentation quite religiously. We deal with an absolute mountain of circuits now that its impossible to expect anyone to remember.

Side note:

When I hired people to replace me at my former work place I used a lab scenario to work out just how clued in they were. I configured a couple of routers and a switch and introduced a couple of errors that meant certain devices could ping each other but others couldnt, and the candidates had to work out why and implement the fixes. One guy took about an hour before I just had to give up and give him the answers (yeah I actually gave him that long...), and another guy worked it out on his own in about 5 minutes.

And when I say a couple I really do mean a couple, 2 to be precise. One was a simple subnet mask mismatch, and the other was a statically configured ARP entry for an IP pointing to a MAC address with only a subtle variation from the original (two swapped characters next to each other.)

I was actually really pleased with the result of this test, it really showed who had experience and how they went about digging information out and checking through things. I would use it again.


accordg

join:2013-01-17
reply to DocLarge

sample config for my office:

Cisco side--
interface Port-channel20
description Storage
switchport trunk encapsulation dot1q
switchport mode trunk
flowcontrol receive on
spanning-tree portfast trunk
spanning-tree bpdufilter enable
spanning-tree bpduguard enable
!

interface GigabitEthernet0/9
description ESX1 - iSCSI
switchport access vlan 100
switchport mode access
srr-queue bandwidth share 10 10 60 20
srr-queue bandwidth shape 10 0 0 0
priority-queue out
mls qos trust dscp
storm-control broadcast level pps 100
storm-control multicast level pps 100
spanning-tree portfast
spanning-tree bpdufilter enable
spanning-tree bpduguard enable
service-policy input qos
ip dhcp snooping limit rate 100
!
interface GigabitEthernet0/10
description ESX2 - iSCSI
switchport access vlan 100
switchport mode access
srr-queue bandwidth share 10 10 60 20
srr-queue bandwidth shape 10 0 0 0
priority-queue out
mls qos trust dscp
storm-control broadcast level pps 100
storm-control multicast level pps 100
spanning-tree portfast
spanning-tree bpdufilter enable
spanning-tree bpduguard enable
service-policy input qos
ip dhcp snooping limit rate 100
!
interface GigabitEthernet0/11
description ESX3 - iSCSI
switchport access vlan 100
switchport mode access
srr-queue bandwidth share 10 10 60 20
srr-queue bandwidth shape 10 0 0 0
priority-queue out
mls qos trust dscp
storm-control broadcast level pps 100
storm-control multicast level pps 100
spanning-tree portfast
spanning-tree bpdufilter enable
spanning-tree bpduguard enable
service-policy input qos
ip dhcp snooping limit rate 100
!
interface GigabitEthernet0/12
description ESX4 - iSCSI
switchport access vlan 100
switchport mode access
srr-queue bandwidth share 10 10 60 20
srr-queue bandwidth shape 10 0 0 0
priority-queue out
mls qos trust dscp
storm-control broadcast level pps 100
storm-control multicast level pps 100
spanning-tree portfast
spanning-tree bpdufilter enable
spanning-tree bpduguard enable
service-policy input qos
ip dhcp snooping limit rate 100
!


aryoba
Premium,MVM
join:2002-08-22
kudos:4
reply to TomS_

said by TomS_:

Haha yes. The test of a true engineer is if they can reverse engineer what already exists.

To top it off, try without any diagrams and without full visibility of the entire network. Even if you can do it, you may be blamed of doing things not by company standard regardless your work quality that may be higher or simpler than the standard itself

aryoba
Premium,MVM
join:2002-08-22
kudos:4
reply to TomS_

said by TomS_:

When I hired people to replace me at my former work place

Why would you want to hire people that will replace you in the first place?


TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5

Because I was leaving.


HELLFIRE
Premium
join:2009-11-25
kudos:18
reply to DocLarge

said by TomS_:

When I hired people to replace me at my former work place I used a lab scenario to work out just how clued in they were.

Wonder how well I would've fared in that test / situation. I wouldn't say I'm a complete network n00b, but I agree with your statement TomS_.

Regards

DocLarge
Premium
join:2004-09-08
kudos:1
reply to DocLarge

Well, it appears there still lies the issue "to document or not to document, that is the question..." in the civilian sector

Being I'm a retired AF communications troop, it was "mandatory" that documentation be accomplished to ensure continuity was in place. While I was active duty, we had numerous contractors try and pull that crap of not wanting to document processes because they "thought" it would give them job security; we typically dismissed anyone who failed to maintain continuity (we just couldn't afford for someone to bring that type of civilian mindset into a warfighting environment).

Personally, now that I'm contracting, I'm applying the same standard in my local environment as well. Whether an individual is better, best, or worst, "somebody" better have it written down (of course, documentation is also listed as a Cisco best practice, so it goes without saying...)

Anyone, thank guys for chiming in. Nosx helped bring some clarity to me after getting me to stop looking at things from an "overly grandular" perspective; all is it good so I've got my starting point...

Jay


aryoba
Premium,MVM
join:2002-08-22
kudos:4
reply to HELLFIRE

said by HELLFIRE:

said by TomS_:

When I hired people to replace me at my former work place I used a lab scenario to work out just how clued in they were.

Wonder how well I would've fared in that test / situation. I wouldn't say I'm a complete network n00b, but I agree with your statement TomS_.

Regards

In some companies, lab test is standard interview process. Just think of it as CCIE mock lab

HELLFIRE
Premium
join:2009-11-25
kudos:18
reply to DocLarge

@DocLarge
...corollary of that is "documentation is from an obscure dialect that roughly translates as 'for some other poor schmuck do to.'"

@aryoba
Haven't done IE, and don't know if I want to... I think I know my stuff pretty well, it's just one of those "now your career
is on the line, prove it!" kind of things.

Regards