VMware server and network setup

rolion

Hi,
Trying to setup/install a new vSphere setup.
Did my homework and i think i might go for this one:

HP ML350
2xCPUs
12Gb RAM
3x300Gb sas drives
3xdual port NICs

All these backed by a good switch that knows trunking,lacp and vlans in order to take advantage of the NIC teaming on the host.

There are two physical networks with around 50 PCs each.

On the host i have 3 to max 5 VMs,mainly Server 2003 for AD,File and Printers. I have converted P2V,but not sure if that is too ok as the existing server OS may have glitches from the Physical life...

What else i'm i missing here ?
Thanks in advance...

qwertz

I am doing a migration to vSphere at the moment. My client is going blade so my scenario is a bit more complex.

I take it that you are going to install the latest version of vSphere. Also, ESX or ESXi? I went with ESXi since VMware are planning on phasing out ESX and some other reasons.

First thing you should do is go to the VMware site and check that the CPUs in your ML350 are supported by vSphere 4.1. Also check all the other components, especially the network cards.

http://www.vmware.com/resources/compatibility/search.php

Firmware is very important as well, make sure you are up-to-date and supported.

How will you backup your VMware environemt? Remember that you will have everything on one box and if this one goes down then you are in trouble. Have a good backup and BC plan in place. It might make sense to install ESXi on one of the older servers and keep a copy of all the VMs on there that you can bring up through scripts or manually.

kippy

Ideally, and this depends on your budget and setup:
To make the most of a virtualised environment you need to identical hosts (at minimum) and some form of a shared SAN to store all the VM's.
This allows for VMotion, failover, faulttolerance as well as any other number of the major benefits of Virtualisation.
Outside of that.
ESXi is defo the way to go, provides a few more features than ESX and as someone has mentioned ESX is getting phased out.
It is most important to ensure that the hardware is supported by VMWare and on their supported list of hardware - you will have issues with support otherwise.

When you are converting P2V, the main issues are if there are any specific "non standard" hardware on the P server that the P server requires (modems/graphics cards?

1 further tip as it is difficult to know exactly what your requirements are.
TEST TEST TEST before going live.

Jumpy

kippy wrote: »

Ideally, and this depends on your budget and setup:
To make the most of a virtualised environment you need to identical hosts (at minimum) and some form of a shared SAN to store all the VM's.
This allows for VMotion, failover, faulttolerance as well as any other number of the major benefits of Virtualisation.

Only if you can afford it.

kippy wrote: »

Outside of that.
ESXi is defo the way to go, provides a few more features than ESX and as someone has mentioned ESX is getting phased out.

ESXi is actually a stripped down version of ESX. It has less functionality, but less attack surface.
If you can live without a service console (which I think I have actually had to use three times in my entire VMWare history + Isnt that what the iLO is for ) then go ESXi.

No point really learning ESX anyway if its gone soon.

kippy wrote: »

1 further tip as it is difficult to know exactly what your requirements are.
TEST TEST TEST before going live.

^ This x1000000

rolion

Thanks to all for reply...

I'm going for HP ML350 G6,so that should be no problem with hardware,as is listed as fuly compatible with VMware.On top of that,i want to get extra DualPort NICs,not sure which one yet...i heard Intel are better 82575 chipset.

I'm getting a DLink switch,create LACPs for the dual port nics,then a small vlan for each server and from there,cascading down to switches for each netwrok.So,if am correct,2x2Gb=4Gb full duplex bandwidth server to switch and then shared over to cascaded switch to 24 - 24 ports on the two switches.

Not sure about SAN,the site is too small for that kind of scenario and if all goes down,one day is downtime acceptable from end-users...i have 4h response time from HP,that shall cover hardware.
Re software-ise,i will copy the VMs and/or snapshots over Network,on to a dedicated storage location,maybe once a week/month.

Re backup,only file sharing services on servers,so maybe CA or Symantec may do the job from within the VMs on to a cheerful NAS box or another physical dedicated backup Server and from there,dumped on a tape or on a 2x2Tb hard drives .

Re software,i think i may go for Vsphere 4.1 Standard,i don't have any requirements (yet) for Vmotion and/or secondary servers to host VMs.

Conversion P2V went smooth,took a lot of man-hours,but they are all up & running,still copying data from shared folder on to the secondary disk on VM Servers...

Now,i'm not sure how to test it before going live !??
What can go wrong ?
Slow network access ?
Slow login times ?
Disconnection,data corruption !?

what are normal/acceptable/mangeable bottlenecks in this kind of virtual environment !?

Thanks in advance...

Jumpy

You have no SAN or iSCSI system as well as no VMotion license so component failure (CPU, network, HDD etc) is your biggest risk.
For example, if you lose that one box, you lose everything.

Do not under any circumstances use Thin provisioning (if you end up getting the required licencing level) on a single array environment.

You will get SCSI reservations every time the disk expands (which at the start is ALL the time). So times this by the number of Thin Disks you have and you have a bit of a bottleneck.

Get a VM aware backup system with De-duplication. It will save you space over time. It also means you can restore systems to an exact point in time. Unlike file system aware backups (you need to rebuild the server first)

Jumpy

rolion wrote: »

Slow network access ?

With 6 network cards on the physical and 4 in the same virtual switch and running the VMXNET3 cards on the virtual machines. This will not be an issue.
You have a small amount of virtual servers.

rolion wrote: »

Slow login times ?

No longer than physical if you have your domain correctly configured.

rolion wrote: »

Disconnection,

Only if you lose the physical switch.

rolion wrote: »

data corruption !?

It happens, but its unlikely. Keep backups. Always.

rolion

thanks,i'm much obliged !

didn't know about thin provisioning,learned soemthing new tonight ! tbh,i used full flat allocation of disk space,so i'm lucky this time...

i'll run 'few tests' tomorrow and see how that goes ! hopefully,all ok !

Jumpy

You mean vmdk?

You are using virtual disks? You are not using raw disk no?

rolion

Jumpy wrote: »

You mean vmdk?

You are using virtual disks? You are not using raw disk no?

sorry...yeah,vmdk ! when i created VMs,it asked me what type of disks i want to use...flat or thin ! selected flat !

tell me something:i want to install a separate RAID controller,maybe a P400
with few SATA drives,3x2tb with 64m cache in raid5,in order to expand storage,as the sas drives are adding as cost same as the bare server hardware ! Do you thing that will slow down the server'disk IO as being sata rather than scsi 10k !?

Jumpy

Yes. The performance will be lower, you should be able to use that area for archive server drives though. Or backup storage space.

If you dont have too many accessing it, then you can use it as File Server space.

Keep your OS on high speed disks though.

There is a reason they are much cheaper

longshotvalue

just having a quick look at this, for your backup Veeam would be a good choice, if you add a small HP ML110 G6 with a 1tb disk, you can use the extra vsphere host license ( i assume you got vmware essentials bundle), to create it as a replication host, and the software will replicate the virtual machines to this host, if there is a problem with one of the vms or the host the backup one can be started. Get 4-8 gigs of ram for the backup server.

Veeam will also do the standard full backups of the virtual machines, to a NAS box..

This solution will give you some of the redundancy of a SAN without the huge cost..

Jst looked at you specs further and Vmware Essentials Bundle along with Veeam should be ideal and cost effective..

rolion

Hi,thanks to all for updates...
Unfortunately,the test pilot didn't work out in my best intention.
My imagination hit the cold wall of reality ! !

Most of the users are having roaming profiles,that can be from 100m to 5gb.Syncronising that amount of data at log on / log off...makes the system to react very slow,minutes ...once in the desktop,most of the work is acceptable speedwise !
So i have to revert to physical setup,undoing all work done from P2V.

Not sure what is causing it,but not doing it well for me...

Also,what i found and drove me mad for a while...in one of the servers,the AMd VMware netwrok card kept the CPU 100% system.exe busy and,until i've replace it Intel 1000 didn't went under 99% ! All ok after change tbh...strange !

Thanks.

kippy

rolion wrote: »

Hi,thanks to all for updates...
Unfortunately,the test pilot didn't work out in my best intention.
My imagination hit the cold wall of reality ! !

Most of the users are having roaming profiles,that can be from 100m to 5gb.Syncronising that amount of data at log on / log off...makes the system to react very slow,minutes ...once in the desktop,most of the work is acceptable speedwise !
So i have to revert to physical setup,undoing all work done from P2V.

Not sure what is causing it,but not doing it well for me...

Also,what i found and drove me mad for a while...in one of the servers,the AMd VMware netwrok card kept the CPU 100% system.exe busy and,until i've replace it Intel 1000 didn't went under 99% ! All ok after change tbh...strange !

Thanks.

Roaming profiles. Nightmare.

I suspect the issue was more with the network bottleneck than VM itself.

rolion

kippy wrote: »

Roaming profiles. Nightmare.

I suspect the issue was more with the network bottleneck than VM itself.

Client Desktop=100m-> switch 100m -=> Central Switch 1Gb managed with Vlan per LAN and 3pNICs in trunk(6Gb FD) -> VMware 1Gb vNIC

Not happy...i may take in consideration to disable roaming profile / offline files per user and start fresh,with a minimal user profile and all personal data stored on a mapped drive from storage server,in windows explorer...not sure,but is worth to check it !

thanks.

longshotvalue

rolion wrote: »

Hi,thanks to all for updates...
Unfortunately,the test pilot didn't work out in my best intention.
My imagination hit the cold wall of reality ! !

Most of the users are having roaming profiles,that can be from 100m to 5gb.Syncronising that amount of data at log on / log off...makes the system to react very slow,minutes ...once in the desktop,most of the work is acceptable speedwise !
So i have to revert to physical setup,undoing all work done from P2V.

Not sure what is causing it,but not doing it well for me...

Also,what i found and drove me mad for a while...in one of the servers,the AMd VMware netwrok card kept the CPU 100% system.exe busy and,until i've replace it Intel 1000 didn't went under 99% ! All ok after change tbh...strange !

Thanks.

The slowness here has nothing to do with vmware, if the roaming profiles are that big, they are always going to cause an issue.

Have you got extra network cards for the vmware , add a 4 port gig network card to the esx server and that should help the network bottleneck

rolion

aidankkk wrote: »

The slowness here has nothing to do with vmware, if the roaming profiles are that big, they are always going to cause an issue.

Have you got extra network cards for the vmware , add a 4 port gig network card to the esx server and that should help the network bottleneck

I have a HP Quad giga pNIC,i'm using 3 for one network and 1 for second lan,less under pressure and importance...that will give me a 6Gb FullDuplex (checked on the switch ,all 3 ports in trunk are Tx/Rx mad...)

I'm going to request that all users login ,just for a day,with a template VM1 and so on,with no roaming profile or offline files and see how that goes.
Also,i will approach the VMware System Engineer for Dublin and get his advise too...

I'll keep you posted ...
TIA

kippy

A five gig profile is a lot to pull down over a 100Meg link........
I dunno, I reckon the size of the profiles are the issue.
Have you many users?

Jumpy

Redirect your profile folders using GPOs.
You shouldnt have profiles of more than 10mb in size.

rolion

:mad:Quick update:removed roaming profile completely...login works fine now,much better ! Waiting for more feedback but at least ,after i fcuked the user profile,i hope it will work better.Few more days and i should have a better picture.

BUT...my overall experience to VMware was quite bad...

1.Reinstalled VMware tools,after i changed from AMD to Intel NIC...it enabled the firewall on the Server,for about 20 minutes i've bang my head in the wall trying to figure why the whole network is down,no DHCP,no DNS,no sharing...no nothing !! MAD,why is enabling the firewall !???

2.The system.exe is keeping the two cores at 99% utilisation,is again MAD !The whole server is slow in responding from Remote Console...Not sure why,i have no ideea and i don't have the fcuink time to debug-it ! If i can't fix it myself,imagine the internal non-it lads...going banana !

3.Performance wise,not impressed...i may go with few switches so i have a full 1gb end desktop to vSwitch,but at this stage i will be very delighted to go back to Physical world completely. Actually ,last night i was 1 second from switching over ,after i checked the bloody firewall on the network connection...


4.Called the VMware system Engineer in Dublin,he said never heard something like that and not sure how to assist me...Surprised,but after going to bed few nights at 2-3am,i hoped that two brains are better than one !Not quite sure...


SO...wtf i'm i doing wrong OR right !?

tia

kippy

rolion wrote: »

:mad:Quick update:removed roaming profile completely...login works fine now,much better ! Waiting for more feedback but at least ,after i fcuked the user profile,i hope it will work better.Few more days and i should have a better picture.

BUT...my overall experience to VMware was quite bad...

1.Reinstalled VMware tools,after i changed from AMD to Intel NIC...it enabled the firewall on the Server,for about 20 minutes i've bang my head in the wall trying to figure why the whole network is down,no DHCP,no DNS,no sharing...no nothing !! MAD,why is enabling the firewall !???

2.The system.exe is keeping the two cores at 99% utilisation,is again MAD !The whole server is slow in responding from Remote Console...Not sure why,i have no ideea and i don't have the fcuink time to debug-it ! If i can't fix it myself,imagine the internal non-it lads...going banana !

3.Performance wise,not impressed...i may go with few switches so i have a full 1gb end desktop to vSwitch,but at this stage i will be very delighted to go back to Physical world completely. Actually ,last night i was 1 second from switching over ,after i checked the bloody firewall on the network connection...


4.Called the VMware system Engineer in Dublin,he said never heard something like that and not sure how to assist me...Surprised,but after going to bed few nights at 2-3am,i hoped that two brains are better than one !Not quite sure...


SO...wtf i'm i doing wrong OR right !?

tia

Have you ever done a course in VMware/implemented it before?
There are a few things to be aware of, as with any major changes and the only way you can prepare for these is:
A. Being familiar with the software (perhaps doing a course in it)
B. Having experience with the softwa.
C. Having an expert do the implementation.
D. Testing testing testing.......

rolion

kippy wrote: »

Have you ever done a course in VMware/implemented it before?
There are a few things to be aware of, as with any major changes and the only way you can prepare for these is:
A. Being familiar with the software (perhaps doing a course in it)
B. Having experience with the softwa.
C. Having an expert do the implementation.
D. Testing testing testing.......

Thanks for help along the road...i owe you one !!
I'm VSP,VTSP ,VCP-=> when i'll make 3k to pay for their 5 days coffee aka training

I've done 4 projects RFT on the paper before,i have 3 VMware servers in my home lab,many Videotrainings Courses,thousands of PDFs...and still learning !!

Re expert,i barely make myself a living,calling an expert to do it for me,what i'm doing myself...not quite sure that will change too much !

Testing ? I am very happy that my end users have gone quite nicely with me along this adventure ! At the end,if they are happy,i'm happy too...and my bank manager too !!

Now,don't take it so badly that i'm pissed off with this implementation,i had a look on the VMTN and there are people with more serious problems than myself...nothing wrong with the product itself maybe,just need exposure,exposure and patience at the end of the day !??

tia

qwertz

rolion wrote: »

nothing wrong with the product itself

I do not concur with this statement. There's plenty wrong with the product itself but most can be worked around. Sometimes it's easier to bite the bullet and pay for an expert.

Make sure you get a real expert, not some chancer! There are numerous ones being sold as experts by the main consultancy companies in Dublin. No, won't name and shame because I don't want to get my backside sued off.

rolion

qwertz wrote: »

I do not concur with this statement. There's plenty wrong with the product itself but most can be worked around. Sometimes it's easier to bite the bullet and pay for an expert.

Make sure you get a real expert, not some chancer! There are numerous ones being sold as experts by the main consultancy companies in Dublin. No, won't name and shame because I don't want to get my backside sued off.

you're so right,i could be one of those consultants maybe,but we all started somewhere,isn't !??

btw...you forgot the 'maybe' from my statement,but..as well,i don't want my backside fired !

my best regards to all of you !

Jumpy

Im a VCP

What OS(s) are you running on the vms?

If the OS supports it use the VMXNET3 adapter. 2003 will support them.

Are you running Hardware Version 7 or 4 ?

VMXNET3 adapters should be added as NEW adapters.

Also if you are getting 100% on windows task manager it means you havent got enough virtual CPU assigned to your VM.

Always use the performance monitoring on your vsphere, dont rely on what windows is showing you. Virtual works differently.


We have over 40 machines running on our cluster with no performance problems.

rolion

Im a VCP
Nice one,congrats...whopaid for the training class !?

What OS(s) are you running on the vms?
All VMs are running Windows Server 2003 with latest SPs and updates,basic File sharing services and one of them with a SQL2003 but a small web based interface as the frontend.

Are you running Hardware Version 7 or 4 ?
All HW is version 7,as done at conversion moment.

VMXNET3 adapters should be added as NEW adapters.
Yes,i realised that,they are vmxnet3,Intel 1Gb.

Also if you are getting 100% on windows task manager it means you havent got enough virtual CPU assigned to your VM.
I have reservations set for three VMs for CPU and Memory,still no joy,system.exe going to 99% cloging one server.

Always use the performance monitoring on your vsphere, dont rely on what windows is showing you. Virtual works differently.
Yeah,i will ,but at the moment i'm more concerned with the system.exe,it should stay under 50% most of th etime for just basic file printing and logins...

We have over 40 machines running on our cluster with no performance problems.
i believe you...

Thanks...

swampgas

Performance problems can be tricky. What if you have just one or two VMs running - do they see the same high CPU usage?

Have you looked at disk IO latency? Try logging into the ESXi management interface and running esxtop, if you look at the disk stats you can see this real time.

Have you looked at /var/log/messages? Any obvious errors?

rolion

As per last white night... TOMORROW BACK TO PHYSICAL or V2P !
had enough...i owe a lot to my end-users for this adventure !

i will list my experience in few days as: "how do not P2V: one way out off a hundred !"

thanks to all... for all.

kippy

rolion wrote: »

As per last white night... TOMORROW BACK TO PHYSICAL or V2P !
had enough...i owe a lot to my end-users for this adventure !

i will list my experience in few days as: "how do not P2V: one way out off a hundred !"

thanks to all... for all.

Sorry about the previous post, I was wondering had you any experience on VM due to a previous post you had.

I am not experienced, nor qualified in anything VM realted.

swampgas

rolion wrote: »

As per last white night... TOMORROW BACK TO PHYSICAL or V2P !
had enough...i owe a lot to my end-users for this adventure !

i will list my experience in few days as: "how do not P2V: one way out off a hundred !"

thanks to all... for all.

Just out of curiosity - do you have a support contract? Did you contact Vmware at all?

Jumpy

Sorry, I thought I had this one on my watch list, but I guess I did not.

What did you use to do the P2V?

Was it cold or live?

Did you remove your old network adapters from the system when adding the VMXNET3s? They become invisible and you have to ensure you remove them properly by making sure you are in advanced mode in Hardware manager in windows.

Is it only the one guest system hogging the cpu? Is everything fine when this is powered off?

Do you have anything on that server that needs direct access to the network card? Remember as I put above, deleting the virtual NIC in Vsphere does not remove it from windows for you.

What is the primary function of the server that is hogging? Can it be clean rebuilt?
This might be both cleaner and less troublesome than going V2P.

Remember not all servers can be virtualised (although this seems more of a glitch than high usage)

Jumpy

Also do you have any virus scanners active?